Deconstructing Video I

Video is just a sequence of images, so I can deconstruct video in the same way that I deconstruct images.  This way, though, we get a movie instead!

https://www.youtube.com/watch?v=OPm290OeN7c

Back To Kindergarden

One of the first times I ever encountered what I'd now call an 'obstacle' occurred with my first two computers.  The very first one was a Timex/Sinclair ZX81.  I remember the day I got it, I connected it all together and didn't stand up again until the next morning when I had to go to school.  I literally stayed up all night learning BASIC.  As I recall, I wasn't tired at all the next day.  Anyhow, I learned how much power I had in my hands programming that simple computer.  I still have the ZX81.  Maybe a year or two later, I somehow was able to get the T/S 2068 with 72k (!!!) memory, which was 72x more than the ZX81 had.  I still have that one, too!

Anyhow, between these two computers, I bumped into assembly code and machine code once in a while.  I had bought a few games (on cassette tape), and had noticed that when I 'listed' the program, it was one line that I can't recall (probably USR and a number), and then a 'DATA' command followed by what looked like a long list of random ascii (and non-ascii) characters.  Somehow this was the machine code, but I was clueless.  I had no understanding of how a computer worked, so I'd have no chance understanding assembly code and much less machine code.  I knew it allowed for 'better' software (like faster, better games), but I had no idea how to even approach it.  I was stumped and felt that first obstacle.

Even when I was a pro programmer and had to deal with assembly code, it was very very minimal and I hardly understood anything even then.  At least by then, though, I'd gained a basic understanding about how computers works.  I even started building a computer based on a 68000 that I'd pulled from an old laser printer (I got as far as wire wrapping, but I never wrote any software for it or got it to run).

A couple of months ago I heard a great interview with a couple of the major developers of ffmpeg and vlc.  They surprised me when they said that assembly code is still very much needed and used in all kinds of things, and that good programmers are very much needed for this.  Ultimately, it'd be pretty cool to be able to contribute to these and similar projects.

Anyhow this got me thinking again about assembly and machine code.  My mind immediately flashed to my ZX81 and 2068.  Finally do some machine code on them?  I probably need a cross compiler.  Well, after several attempts, none of the linux cross compilers I tried generated code that could be run on an emulated ZX81.  Oh well, but since ffmpeg and others are written for x86 (and other) processors, why not just work with assembly for the x86 instead of the z80?  Good idea, but after a couple of weeks I haven't really made any progress on that.  These 'modern' CPUs are very complex -- even more so that the Z80.  Writing assembly code for modern CPUs is a pretty big bite.

So I've decided to jump back on to the T/S 2068 and learn Z80 assembly on it.  Z80 assembly is hardly simple, but it's much less complex than x86 or anything else.  I don't need an external cross compiler since I can (and will) do it manually.  I'll have these things memorized pretty quickly probably.

The T/S 2068 has a 'Monitor' out RCA plug, which is probably composite video.  I got a composite video to VGA adapter (amazon, $12, 2 days), and plugged everything in for the first time in at least 10 years on this 44 year old machine!  Flipped the power switch and....

So wow yeah it still works!

I've already started writing some MC on it, but very very basic stuff.  I crashed it several times likely due to incorrect programming.  Even a fail at this point tells me that I'm tapping into that great unknown.

The only thing I really need to do is make sure I can save and load programs to and from 'tape'.  That's today's task.  That, and continue to learn this.

I have a project in mind that I'd love to write for the T/S 2068.  Once I graduate kindergarden I'll be able to graduate to more complex programs.  Too early to say anything more about it for now.  But hopefully soon I'll be able to start thinking about how to do it.

Satisfying a childhood dream that has actually felt more like an unfinished project.  In any case, it's nice to be able to finally do this even though the leaning curve is gong to be steep and bumpy.  It'll be fun, and it'll get me thinking in computer-ese so when I move over to x86, etc., I'll be more comfortable diving into unknowns.

Bee Motion I

I swarm of bees decided to honor my home with a surprise move-in.  Before I had them removed (gently and professionally), I grabbed some video (about an hour total) of their flying motion.

I can then compute the differences between images and then sum those differences into a 'long exposure'.  If I set a threshold correctly in my code, the motion of the bees just pops out.

Bees move very fast so this is a short 2 second exposure running at 30 Hz.  So this is the sum of a the differences between 60 contiguous video frames, thresholded so only the largest motions (the bees) shows up:

They are true artists and flying machines, or they really have no idea what they're doing.

Here's another 2 second sample:

Deconstructing Images III

Was going through some old photos, and came across this.  Dawn 10 Feb 2018:

I was impressed not only by the colors, but also the textures.

Anyhow, here's a combination of 'color-magnitude' diagrams put together in an artsy way, but using the colors you see in the image above.  In all my work, the data will always dictate.

The structure on the furthest left is B (x-axis, 0 to 255 left to right) vs R-G (y-axis, 0 to 255, bottom to top).  The next left structure (sort of attached to the first structure) is G vs R-B.  The structure bottom right is R vs G-B.

Deconstructing Images II

 Using my previous post's first image of the prickly pear cactus flowers, I'm now generating these "color-magnitude diagrams" using the color camera RGB values -- effectively a three-channel imaging spectrometer.

These are (from left to right): red versus green minus blue, green versus red minus blue, and blue versus green minus red.  Color value goes from left to right, 0 to 255.  Difference value goes from bottom to top, -255 to +255.  Most of the color differences are positive in this particular case.

These images are different that the previous because the pixels in these images have been weighted by the number of pixels corresponding to a particular color and color difference.

Deconstructing Images

 I start with a nice input image of some flowers on my prickly pear cactus:

I then generate a "color-magnitude" diagram for some configurations:

The left two panels show R on the x-axis, and G-B on the y-axis.  The white line separating the two panels vertically represents a value of zero.  So pixels appearing above the white line have a positive G-B value, and those below the white line have a negative G-B value.  The middle two panels show G versus R-B.  The right panels show B versus R-G.  The colors of each pixel correspond to their colors in the input image.  The x-axis range is 0 to +255 (left to right).  The y-axis range is -255 to +255 (bottom to top).

Here's another example, using the image of the Earth taken by Artemis II:

Here's R vs G-B, G vs R-B, and B vs R-G

clearly picks out landmasses, ocean, clouds, and aurorae!

Sandhill Cranes Long Exposure

30 second long exposure of a flock of Sandhill cranes 'circling' over my location a couple days ago.  The cranes are here from mid-October to about March every year.  I'm situated right over their daily flight path as they visit various playas and other locations with standing water this time of year