Uncategorized

Macro focusing rail v2

Uncategorized
Stack of 128 frames of a wasp using a 3.7x 0.11NA objective on a Pentax K-1

Focus is often used creatively in photography to separate the subject from the background of an image. In microscopy ‘optical sectioning’ to resolve details out of the plane of the image. In macrophotography, however, we often want to capture images which are pin-sharp front to back. Doing this is quite hard.

The depth of focus is very narrow at high magnification. In fact, in the wasp portrait the depth of focus was only 58 microns, or only 0.058mm. You can see what a single image looks like bellow. Only few hairs are in focus. In total for this photo I took 128 images in 20 micron steps.

Single frame from stack

It’s pretty hard to move in 20 micron steps by hand, so for a little while I’ve been putting together a focusing system.

Version 1 was just a z-axis rail for a CNC machine. A rail with a carriage supported on it, with a screw thread and a like-threaded insert on the carriage, and a stepper motor. This was controlled by an arduino and stepper driver. The camera was set up with an interval timer, and the arduino code had periods in it for the camera to take a photo.

This setup had several disadvantages, securing the camera to the camera was difficult. an M5 to 1/4″ bolt was used, but this didn’t allow for the camera to be securely fastened. Also, the minimum step size was ~50µm which wasn’t fine enough. Lastly, the camera needed to stay in sync with the arduino, which was achieved by starting the arduino code a few seconds before the camera, not ideal.

Version 2 has a number of improvements. By cannibalising a shutter release cable I’ve been able to control the camera from the arduino, by just bringing a pin high. I also drilled out a tripod base plate to give sufficient clearance for the camera plate to slide into it, while everything is bolted together. Lastely, I swapped out the threaded rod for a M8 fine pitch rod. This rod has a pitch of 1mm, and only 1 thread cut into it, instead of the ~2.5mm pitch and 4 threads cut into the rod I was previously using. This improves the stepping precision by a factor of 10. A single step on the new system is only 5µm, which is only about 10 wavelength of light.

The thread was cut into a small block of wood which was pre-drilled with an 8mm hole. The wood offers quite a lot of resistance, but also doesn’t produce any backlash.

1:1 zoom of the eye of the wasp. 1px is 1µm in size. The facets of the compound eye are most interesting as they transition from the hexagonal packing into a square packing.

JPEG artifact removal with Convolutional Neural Network (AI)

Uncategorized

JPEG images, at high compression ratios, often have blocky artefacts which look particularly unpleasant. JPEG images look particularly bad around the sharp edges of objects.

There are already ways to reduce the artefacts, however, these methods often don’t use very sophisticated techniques, only using the information from adjacent pixels to smooth out the actual transition between the blocks. An example of this can be found in Kitbi and Dawood’s work https://ieeexplore.ieee.org/document/4743789/ which gave me the original inspiration for this.

An alternative way is to use a convolutional neural network (CNN) to intelligently estimate an optimal block given the original block and the eight surrounding blocks. And then tile over the image to estimate the optimal mapping for each block.

The network design was a 5 layer fully convolutional one, using only small filters. Several different architectures were used, which all gave largely similar results. The compromise between effectiveness and speed (i.e. 1/size) was found with a small network with only 57,987 parameters. Training the network was surprisingly fast, taking only a few hours without a GPU.

The network takes in full colour information and outputs the full colour too. The reason for this is to use all of the possible information. The colour channels are highly correlated with one another. It would be possible to train the network on monochrome images, but that would lose the relation which naturally exists.

So, does it work?

In my opinion, yes it does work. I think my method performs best when there are complicated edges, such as around glasses, or on hairs which are resolved as partial blocks. It works least well, in comparison to the method which photoshop currently uses, when there are large smooth areas.

Text was not present in the training data set. So the poor effectiveness of the network on text is not a significant point of comparison. The above images are quite small. A larger example is below, along with a zoomed in video comparing the original, JPEG, photoshop artefact removal, and my method.

I did calculate root mean squared error values one point comparing the network performance with the JPEG image to the original. In some cases the network was reliably out performing the JPEG – which is impressive, but not too surprising, as this was how it was trained. I don’t think that those sorts of values are too important in this case. The aim isn’t to restore the original image, but to instead reduce the visual annoyance of the artefacts.

If you really wanted to reduce the annoyance of the artefacts you should use JPEG2000 or else have a read of this paper https://arxiv.org/pdf/1611.01704.pdf