Blog

Film scanning with Sony mirrorless and Adobe Camera Raw —

I’ve been shooting film for a few years now, and scanning/digitizing it myself almost as long. Here’s a guide to how I do it. This is mostly for slide film but should apply to any types of film.

Equipment

Needed equipment

Distance

You’ll need to start by setting your camera at the right distance from the film. Point it at the film and raise/lower the tripod until it fills the frame. For 6x6 like I have here it will be square, for 135 it should fill the whole frame. Since this is approximate you can leave the film in the sleeve. If you can turn on grid lines on your PC app this can help here. Your film should go between the light pad and the dark mask.

Setting up for distance

View for distance in the camera app

Alignment

To make sure everything is in focus, you need to have your camera perfectly parallel to the film. An easy trick for this is what the mirror is for. Place the mirror on your light pad and align your camera so the reflection of the lens is exactly centered. A grid helps here too. You may need to do several iterations of distance and centering to get everything perfect. Once it’s set up though you don’t need to touch it unless you change film formats (e.g. 135 to 120).

alignment setup

alignment

White balance

You’ll need to calibrate your camera for the backlight you’re using. Take your film out and shoot a picture of just the backlight.

white_balance

Here I’ve placed several color samplers across the diagonal. Set the white balance temperature and tint so the red, green, and blue are equal (pure white). Also set your lens corrections so the lens falloff is compensated for (brightness is equal across the frame). Depending on your lens you may not able to fully compensate for the falloff. You can also do this by looking at the color histogram in the upper right, get the peak as narrow as possible and eliminate the color fringes.

Exposure settings

Put your film in, and hold it down with the mask and whatever you’re using for weight.

Your setup probably looks something like this:

20211202_210052

For exposure settings, use the lowest base ISO your camera supports (probably ISO 100). Avoid any trick ‘pulled’ ISOs.

When it comes to aperture, shoot several test shots and see what looks best. You’ll have to balance sharper apertures with diffraction softness. Here’s a spread at f/2.8, f/4, f/5.6, etc. from the center of the frame.

aperture_center

It’s hard to say but either f/5.6 or f/8 looks the best, here’s the corner:

aperture_corner

I think f/5.6 looks the best so let’s pick that aperture.

For shutter speed it’s a bit of artistic choice. I compared what I saw on the monitor to what the slide looked like on the light pad. It will also depend on the particular photo as you can compensate for over/under exposure in the scanning process.

I ended up using f/5.6 at 1/6 ISO 100 for 6x6 Fujifilm Velvia 50. Don’t forget that you will have to change the exposure for different formats as macro lenses get darker when focused to close distances.

Mess with the exposure slider in Camera Raw then set that difference on the shutter speed on the camera.

Camera Raw settings

This is also a bit of artistic choice but I found leaving everything as neutral as possible looked the best. Set all the sliders to zero as a base setting, and save that as a preset.

exposure_settings

For image settings, generally you want to leave it at the base camera resolution. I turn it down a notch when scanning grainy film like Kodak Tri-X 35mm because the extra pixels aren’t getting you any more resolution.

To save disk space and processing time I usually leave it at 8-bit sRGB, you may want to set it to 16-bit if you take it straight into Photoshop and expect to do some heavy editing. Don’t use Adobe RGB or any other color spaces unless you can control everything all the way up to printing because any slip-up and your colors will look like crap.

You can also crop it here to square or whatever other format you may be shooting.

Tweaking

Really the only tweak I do in Camera Raw is the recovery slider. It will recover blown highlights that are overloading channels. Notice that the bright highlights on the hut are starting to turn glaring yellow and look like pizza cheese or something. There is also a colored triangle in the upper right of the histogram to let you know the highlights are overloaded.

blown_highlights

Add a bit on the recovery slider to fix these highlights. Don’t use too much or your picture may end up looking dull and flat. If you find yourself adding too much, speed up the shutter speed on your camera to reduce the exposure and try the scan again, instead of trying to fix it in Camera Raw.

recovered_highlights

B&W negative

This guide has been for color slide film, which is probably the easiest to scan even though it has a reputation for being hard to shoot. For black and white negative the process is the same except you will need to reduce your exposure as B&W negative is much ‘thinner’. You can also do the inversion here by going to the curves tab. Flip the end points to the opposite side to invert it. You probably also want to add a curve to make things look more natural, here’s a sample of a curve I used.

bw_inversion_curve

This is also artistic choice so play around with it until you get something you like. Since negative film is much more forgiving of misexposure you’ll probably need to tweak the curve for each shot.

Color negative

I have shot very little color negative. You can scan it this way too, but you will need to do the color inversion yourself. Photoshop has some curves presets for this, but I found it pretty difficult. I know there are programs out there dedicated to the color negative inversion process. You might try them out but I have no advice there since I’ve never used them.

Lab scans

I used to have my film lab scan all my slides. They were fine, but I didn’t like the lack of control over how it looked (unless I took my film back to the lab for rescanning). If you decide to use lab scans with slide film, at least ask them to not do any color correction. If you’re shooting with filters they’ll probably color correct the filters right out!

It’s much more of a pain but I’ve found the advantages of home scanning to be:

Comparison

So here’s a comparison between a lab scan and a home scan of the same slide.

Lab:

RVP50_300780011

Home scan:

RVP50_00041

Your opinion may vary but while the lab scan has better resolution and is sharper, the home scan is properly aligned, has better shadow detail, better color, and doesn’t crop off the edges.

Happy scanning!

Optimizing NVMe performance with dm-crypt —

I recently switched over from Windows 7 to Debian Linux on my main desktop, and along the way decided to set up full disk encryption with dm-crypt because it seemed like support was good, so why not? At the very least it would save me any worry if the drive dies and I had to throw it out. In general I would like to have ‘performance’ more than ‘paranoid’ settings when it comes to tweaks. I basically accepted Debian’s default options on the install and went for it. I also ended up doing dm-crypt-on-LVM for the flexibility of multiple partitions. The drive ended up looking like this:

nvme0n1                               259:0    0  1.8T  0 disk  
├─nvme0n1p1                           259:1    0  512M  0 part  /boot/efi
├─nvme0n1p2                           259:2    0  488M  0 part  /boot
└─nvme0n1p3                           259:3    0  1.8T  0 part  
  └─nvme1n1p3_crypt                   254:1    0  1.8T  0 crypt  
    ├─m2-root                         254:3    0   32G  0 lvm   /
    ├─m2-swap                         254:6    0   32G  0 lvm   [SWAP]
    └─m2-home                         254:7    0  1.5T  0 lvm   /home

The problem came when I started running IO-heavy workloads, generally loading large AI models from disk. These only ran at about 1000 MB/s, far short of the 7450 MB/s they promise on the box. I know Samsung probably lies on these benchmarks but not that bad. Unfortunately I can’t remember what I got when testing under Windows 7.

I turned on all the performance options in /etc/crypttab, so the line for the NVMe ends up looking like:

nvme1n1p3_crypt UUID=21d6f689-0bcf-4c4e-ae81-0181c74bcd2f none luks,discard,no-read-workqueue,no-write-workqueue

Again, no saved benchmarks there but there wasn’t a noticeable improvement. Here’s what they looked like afterwards (from kdiskmark, 1 GiB test):

[Read]
Sequential   1 MiB (Q=  8, T= 1):  7096.145 MB/s [   6929.8 IOPS] <  1145.94 us>
Sequential   1 MiB (Q=  1, T= 1):  1130.209 MB/s [   1103.7 IOPS] <   905.32 us>
    Random   4 KiB (Q= 32, T= 1):   734.142 MB/s [ 183535.6 IOPS] <   173.98 us>
    Random   4 KiB (Q=  1, T= 1):    78.616 MB/s [  19654.2 IOPS] <    50.46 us>

[Write]
Sequential   1 MiB (Q=  8, T= 1):  1299.046 MB/s [   1268.6 IOPS] <  6146.53 us>
Sequential   1 MiB (Q=  1, T= 1):  1093.343 MB/s [   1067.7 IOPS] <   854.56 us>
    Random   4 KiB (Q= 32, T= 1):   413.172 MB/s [ 103293.0 IOPS] <   308.34 us>
    Random   4 KiB (Q=  1, T= 1):   210.846 MB/s [  52711.6 IOPS] <    18.76 us>

Note that the queue depth 8 sequential test actually comes pretty close to the advertised speed, but the Q=1 test is more like what I was seeing when running actual large loads from disk.

For comparison I ran a (256 MiB) test on the /boot partition, which is outside the dm-crypt volume:

[Read]
Sequential   1 MiB (Q=  8, T= 1):  6863.149 MB/s [   6702.3 IOPS] <  1164.45 us>
Sequential   1 MiB (Q=  1, T= 1):  3559.749 MB/s [   3476.3 IOPS] <   285.60 us>
    Random   4 KiB (Q= 32, T= 1):   971.108 MB/s [ 242777.2 IOPS] <   131.33 us>
    Random   4 KiB (Q=  1, T= 1):    66.923 MB/s [  16730.9 IOPS] <    59.35 us>

[Write]
Sequential   1 MiB (Q=  8, T= 1):  4557.689 MB/s [   4450.9 IOPS] <  1631.63 us>
Sequential   1 MiB (Q=  1, T= 1):  2848.125 MB/s [   2781.4 IOPS] <   250.41 us>
    Random   4 KiB (Q= 32, T= 1):   280.971 MB/s [  70242.7 IOPS] <   453.73 us>
    Random   4 KiB (Q=  1, T= 1):   264.417 MB/s [  66104.4 IOPS] <    14.64 us>

The read speeds are mostly the same, except that important sequential Q=1 benchmark is over 3 times faster! I did a bit of looking around, CloudFlare had a good writeup on optimizing disk encryption speed, and I also found this useful Reddit post. It seems like the sector size dm-crypt is using is important. It may be using 512 bytes, even though the overlying file system (ext4 in this case) is using 4096 so we never use any less than that size. Let’s check what size nvme1n1p3_crypt is using:

# cryptsetup luksDump /dev/nvme0n1p3
[...]
Data segments:
  0: crypt
    offset: 16777216 [bytes]
    length: (whole device)
    cipher: aes-xts-plain64
    sector: 512 [bytes]

Keyslots:
  0: luks2
    Key:        512 bits
    Priority:   normal
    Cipher:     aes-xts-plain64
    Cipher key: 512 bits
[...]

We’re using a sector size of 512. It’s also using AES-256, in XTS mode (which is why the key is 2x the size).

I set up a test using my secondary NVMe. This is older and slower (Samsung 980 Pro vs the 990), as well as in the 2nd M.2 slot which means it’s sharing 4 PCIe lanes with the network interfaces, SATA devices, etc. instead of its own dedicated 4 lanes to the CPU.

I formatted this guy up, using the above options, but AES-128 to see if there was any appreciable difference. This is the sequence of commands. Note I’m reading the password from a file (which itself is on the encrypted /home drive).

# cryptsetup luksFormat --key-size=256 /dev/mapper/nvme1n1p2 < /my/password/file
# cryptsetup open /dev/nvme1n1p2 test_crypt < /my/password/file
# cryptsetup refresh --perf-no_read_workqueue --perf-no_write_workqueue --allow-discards temp_crypt
# mkfs.ext4 /dev/mapper/test_crypt
# mount /dev/mapper/test_crypt /mnt/test/

Results:

[Read]
Sequential   1 MiB (Q=  1, T= 1):  1126.740 MB/s [   1100.3 IOPS] <   908.13 us>

A little faster, maybe from the smaller key size, but largely the same as above. Let’s try reformatting it with a block size of 4096 using the --sector-size=4096 option when calling cryptsetup luksFormat. Results:

[Read]
Sequential   1 MiB (Q=  1, T= 1):  1994.320 MB/s [   1947.6 IOPS] <   512.92 us>

Nearly a 2x speed-up versus the original case! Obviously this is the way to go. For experimentation I tried a sector size of 4096, but with AES-256 to see how much of an effect that has:

[Read]
Sequential   1 MiB (Q=  1, T= 1):  1768.740 MB/s [   1727.3 IOPS] <   586.19 us>

A little slower, but still faster with the larger sector size.

So, let’s change the key size and sector size. Thanks to the wonderful cryptsetup, we can actually do this online, on the boot drive, while using it! It even tolerates interruptions (not that I wanted to test this, it’s scary…). The command we want:

# cryptsetup reencrypt --key-size=256 --sector-size=4096 /dev/nvme0n1p3
Enter passphrase for key slot 0: 
Auto-detected active dm device 'nvme1n1p3_crypt' for data device /dev/nvme0n1p3.
Finished, time 51m22s, 1862 GiB written, speed 618.6 MiB/s

After an fstrim, a reboot, we end up with:

[Read]
Sequential   1 MiB (Q=  8, T= 1):  7251.646 MB/s [   7081.7 IOPS] <  1121.95 us>
Sequential   1 MiB (Q=  1, T= 1):  1975.299 MB/s [   1929.0 IOPS] <   519.47 us>
    Random   4 KiB (Q= 32, T= 1):   743.511 MB/s [ 185877.8 IOPS] <   171.88 us>
    Random   4 KiB (Q=  1, T= 1):    84.115 MB/s [  21028.8 IOPS] <    47.20 us>

[Write]
Sequential   1 MiB (Q=  8, T= 1):  3065.347 MB/s [   2993.5 IOPS] <  2532.55 us>
Sequential   1 MiB (Q=  1, T= 1):  2050.924 MB/s [   2002.9 IOPS] <   413.80 us>
    Random   4 KiB (Q= 32, T= 1):   477.777 MB/s [ 119444.3 IOPS] <   266.06 us>
    Random   4 KiB (Q=  1, T= 1):   244.687 MB/s [  61171.9 IOPS] <    15.65 us>

Alright so not quite double, but an improvement all around, for free! Write performance was also greatly increased. Still short of the unencrypted performance, but still pretty quick.

In summary, if you’re using NVMe drives with dm-crypt, for max performance I would:

The security implications of the above are left as an exercise for the reader, but personally I’m OK with it.

Banned from Twitter —

I have been banned from Twitter for posting excepts from an article on the polio vaccine, which was published in the Journal of the Royal Society of Medicine.

Oh well it was fun well it lasted, F for my tweeter 2013-2021, another casualty of the ‘rona.

If you are here because you clicked on the link in my profile, you can view the waifu site at chihaya.moe.

Dead site and revival —

It’s been over a year since my last post. I suppose it’s sort of neglect, but I’ve not had much that I feel like writing up in a blog post. I am quite active on my Twitter and Twitch stream, so you could follow those.

I’m thinking about revamping the site though and sort of transitioning formats. I have a lot I’d like to write up in a sort of static article/write-up format that wouldn’t fit well in a blog post. It’s going to take a little bit of rework of the code to get it done though. I think I’ll also remove the comment functionality, since it’s been virtually unused (only 2 comments in over 4 years).

Even more cartoon books —

Wow more manga opinions nobody cares about.

I finished reading K-On! (けいおん!), K-On! College (けいおん! college), and K-On! Highschool (けいおん! highschool). It’s a fun light-hearted series that I really should have read earlier. The premise is simple, a bunch of high schools girls form a music club and get up to the usual anime antics. Anyway you’ve probably heard of it, and it’s popular for a good reason so go read it!

This month a pre-order I placed months ago for a figurine arrived, along with a couple of others I ordered at the same time. There’s some pictures on my Twitter if you care to see them. Worth it I guess, I’ve got the money. I was surprised nobody I knew passed judgement for gay-ass anime girl figurines.

I also finally got around to watching THE IDOLM@STER MOVIE: BEYOND THE BRILLIANT FUTURE! (THE IDOLM@STER MOVIE 輝きの向こう側へ!). It was decent I guess, but the story was nothing really enthralling. It felt more like they just shoved a couple of episodes of the anime together.

There’s another couple of currently-publishing mangas I’m reading:

My Senpai is Annoying (先輩がうざい後輩の話) is a cute little heart-warming series that’s being published on Pixiv. It’s one of those tsundere almost-romance stories.

Wake up please, Ms. Kusakabe (起きてください、草壁さん) is great if you like sleeping and girls that like to take naps.

I watched Pop Team Epic (ポプテピピック) as it was airing. If you can’t appreciate some really avant-garde surrealist humor you might not like it. I thought it was pretty good. A lot of crazy meta-jokes. When I first heard about it (through physical advertisements in Akihabara!), I’ll admit I had no clue how they were going to make it into an anime. But they did somehow and it turned out pretty decent. They gave these guys 24 minutes to make whatever they wanted, and it turns out it’s 12 minutes of whatever they wanted but twice. Go give it a shot, you might like it.

Apart from that I’ve just been up to the usual stuff: work and counterstrike. Follow my Twitch channel if you like to watch people play video games instead of playing them yourself. I’ve also got a vacation planned in a couple of weeks to Las Vegas/Grand Canyon/the Southwest, so maybe that will be fun.

More cartoon book reviews —

I wanted to write an update on the stuff I’ve been reading so…

I finished reading the main series of Nichijou (日常). It’s a great light-hearted comedy manga, but if you can’t appreciate the absurdist kind of humor it features you might not like it, it took me a bit to get into it. If you’re looking for something fun to read I’d totally recommend it. I was something I enjoyed reading during those isolated evenings down in the Mojave desert. The manga almost seems like it was written to be made into an anime with how animated it is. I haven’t watched it though, maybe sometime soon. Helvetica Standard (ヘルベチカスタンダード) is like the DVD bonus features for the manga, I picked it up and you should too if you liked it.

I finished all six volumes of Mysterious Girlfriend X (謎の彼女X ‘Nazo no Kanojo X’) as well. It’s a bit of a longer read but if you wanted to read a romance manga I’d absolutely recommend it. Unlike some others I’ve read I liked how it set the story up for drama but instead of things turning bad everything resolves in a nice wholesome manner, it was almost refreshing. The story was kind of slow and doesn’t make a lot of progression so if you wanted a riveting story that keeps you going for the next volume you might not like it. On the other hand, if you want a sweet, loving, fun story to read you’ll like it as much as I did. There’s an anime too but I haven’t seen it. Considering how many volumes of manga there were I would imagine they cut out quite a bit.

I’m up to volume 4 of Aho Girl (アホガール). I’ll admit I’m only reading it because of the memes. If you want a fun 4-koma comedy manga it’s pretty good.

I’ve been keeping up with Please Tell Me! Galko-chan (おしえて! ギャル子ちゃん ‘Oshiete! Galko-chan’), nothing to add from my last post. Volume 11 of No Matter How I Look at It, It’s You Guys’ Fault I’m Not Popular! (私がモテないのはどう考えてもお前らが悪い! ‘Watamote’) was good too.

K-On! (けいおん!) is another series I started reading. It’s a nice fun little manga with cure girls doing cute things. It’s only four thin volumes so it’s not terribly long.

I also picked up a bunch of doujins at Comiket 93 when I was in Japan over Christmas. It was super cool but I don’t really have any reviews to write. X1001977x.jpg

HTTPS —

I got HTTPS working on all my website thanks to Let’s Encrypt! Props to them for being free and painless (except trying to figure how to get this to work with vhosts, that’s my fault).

Photo gallery —

Since I’ve been getting into photography I’ve put some of my favorites in a gallery go check it out here.

Chinese cartoon books —

I wanted to do a writeup on the mangas I’ve been reading. Since I’ve been spending most of my time in the Mojave desert (what I call the shit-ass desert), I’ve had spare time to read these. As much as I’ve been willing to carry in my backpack anyway, but I’ve got good at traveling light.

I’ve been reading No Matter How I Look at It, It’s You Guys’ Fault I’m Not Popular! (私がモテないのはどう考えてもお前らが悪い! ‘Watamote’) since the English release started in 2013 and it’s been as entertaining as ever. Whether you consider that a good or bad thing, but it’s consistent. Funny but tough to read at times since the authors are so good at getting that cringy feeling. My only hope is that they come to a satisfying conclusion, better than petering out or choosing some cop-out ending.

I’ve been keeping up with Please Tell Me! Galko-chan (おしえて! ギャル子ちゃん ‘Oshiete! Galko-chan’) and it’s great. Genuine, funny and entertaining, my only concern is that the covers are way too sexual so someone looking at it might get the wrong idea (look at this).

I picked up Nichijou (日常) for reasons I can’t remember. It took me a bit to get into its absurdist sense of humor but it’s pretty funny once you get past the expectation that it’s supposed to make sense. It’s something fun to read at the airport.

I made it through 3 volumes of Haganai: I don’t have many friends (僕は友達が少ない ‘Haganai’). It was OK but I couldn’t really get into it. Maybe once I’m done reading what I am now I’ll pick up the next volume.

Kokoro Connect (ココロコネクト) I read the first volume of but couldn’t get into it either.

I picked up the first volume of Your Name (君の名は ‘Kimi no Na wa’) but I haven’t read it yet since I think I should watch the movie first. I’ve heard it’s really good so I’m looking forward to it. I’ve really liked Shinkai Makoto’s previous films. ‘Looking forward to it’ but the only reason I haven’t seen it is due to my laziness lol.

I read the manga version of 5 Centimeters Per Second (秒速5センチメートル ‘Byousoku 5 Centimeter’) . It’s quite a bit longer than the anime and expands the story a couple of chapters. Maybe it’s because I’m in a different place in my life now, or maybe because the manga expanded a lot on the anime, but it hit me a lot harder emotionally. I’d really recommend it. Watch the anime first for the beautiful visuals, but read the manga for the story. The final chapter really made it, a much more positive ending, or at least more ambiguous and not as negative.

On the recommendation of PaperChazz I’ve been reading Mysterious Girlfriend X (謎の彼女X ‘Nazo no Kanojo X’). I’d agree with him that it’s a really great romance manga. If you just read the synopsis of the story it might seem like this weird drool-centric story, but it’s really great. I’m on volume 4 right now. If it seems interesting to you go check it out.

That’s all I can think of for now.

Things —

I have a new domain name: chihaya.moe. Something coming there soon.

I’ve been down in the Mojave desert most of the time.

This sucks.

Follow my twitter for more frequent updates @crazyfarmsexy.