Illustrating the complexity of silicon (Deep zoom of a CPU)

[SpiralElektronik]

Hi all,

Ever since I was a little kid I have always wanted to see what silicon actually looks like. Of course there are loads of beautiful die-shots out there but none that really captures the immensity of a big CPU or similar. So three years ago I started my slightly idiotic journey of trying to capture the immensity of a silicon die in  photo and ultimately video. I choose an intel 486 DX2 as my subject. In part because I had one lying around, it is easy to expose the die etc. But also because the feature size of 1 µm meant that I could capture every detail of it and still capture the entire silicon die. I have tried to make this illustration as realistic as I possibly could, I used 3D software but all of the textures are actual photographs and the color grading done is minimal.

If you want to see what an old 486 DX2 cpu looks like take a look here:

[free_electron]

most of the people that made that are now retired and the design files are stored in some archaic format that nobody can read anymore. The computer systems , hardware and design software are long gone.

[T3sl4co1l]

Great shot!  Yeah, really blends well between photo and magnifier.  There was just that motion blurred? frame at the start of the zoom which stuck out to me, very nice overall.

Occurs to me, if you could zoom just a little bit deeper -- and actually photograph the flyby, not just simulate it -- this was just a 3D projection of the flat image, I expect? -- I can just imagine how much detail you'd see, for the thickness and depth, and maybe even the refraction, of the glass layer; they really did quite a good job with aspect ratios, and there's real height to that construction, it's only flat because there's so much of it.  That, and the iridescence from polarization, reflection angle, angled path and just any sources of diffraction, would make a really cool scene.  I suppose it'd be a PITA to set up a microscope to do all that, but imagine...

The other thing that came to mind, the broken edge was a bit of a Chekov's Gun to me -- was there anything interesting visible on the corner?  I can just imagine using that as a diving-in point to show, like, a 3D cross section of the circuitry, but oh man, how much work would that be?! Or transitioning to a matched perspective SEM (after etching away the glass), say.

Well, not to give you too much work or anything, but suffice it to say it was thought-provoking, cheers.

Tim

[mawyatt]

Nice video 

Keep in mind that with todays SOTA CMOS processes one can integrate an entire 486 equivalent processor under a single bond pad...amazing

Best,

[HuronKing]

I'm sure this won't generate a ton of discussion but this is cool.

[SpiralElektronik]

Occurs to me, if you could zoom just a little bit deeper -- and actually photograph the flyby, not just simulate it -- this was just a 3D projection of the flat image, I expect? -- I can just imagine how much detail you'd see, for the thickness and depth, and maybe even the refraction, of the glass layer; they really did quite a good job with aspect ratios, and there's real height to that construction, it's only flat because there's so much of it.  That, and the iridescence from polarization, reflection angle, angled path and just any sources of diffraction, would make a really cool scene.  I suppose it'd be a PITA to set up a microscope to do all that, but imagine...
Tim

The main image that this video is based consists of over 700 dslr exposures. Taking the pictures in order without missing any part of the chip was time consuming and difficult. Stitching them together perfectly ended up taking a couple of years of hard work.  The final result is an image that is 3.3 gigapixels or approximately 50kx70k pixels. A microscope has a very limited depth of field so doing it the way you suggest would mean 100s of exposures for each frame. You wouldnt need to take 700 exposures for each fram though since the whole chip wouldnt be visible, more like 30 exposures in the x direction and perhaps 5 (?) in the y direction. That would mean that the raw photos would need 100x30x5x60 exposures per second of animation or very roughly a million exposures per second of video. It would take a cluster of good computers and months to put it al together. Perhaps petabytes to store the raw data.

Interresting thought experiment but I gather Ive spent enough energy on this already... It is really an example of a sunken cost fallacy on a quite large scale. Edit: Thinking some more about it, it would be feasible to do a really close up shot, closer than closing scene the way you describe. The focus stacking would be the difficult part to pull of, but with automation, anything is possible. My calculation above is of course off by an order of magnitude.

The cracked chip is due to me slipping with the chisel when I (crudely) opened the chip.
I find the scratch i made with the chisel a little bit interresting, since it shows the plastic deformation of SiO2, and also has the features of transistors clearly visible 
 
I originally planned to make a longer format youtube video explaining how CPUs work in general and how this architecture works in particular but I don't think that is likely to happen now.

If you want to see more images of something in particular please let me know.

Heres an image taken with even larger magnification (100x) where you can make out the 3d shapes of things sort of:

[magic]

Crazy stuff

The owner of Zeptobars had an ambition to automate "scanning" of large ICs and fully reverse engineer the i386
But it looks like "real" work has been pulling him away from the hobby in recent times.

[T3sl4co1l]

The main image that this video is based consists of over 700 dslr exposures. Taking the pictures in order without missing any part of the chip was time consuming and difficult. Stitching them together perfectly ended up taking a couple of years of hard work.  The final result is an image that is 3.3 gigapixels or approximately 50kx70k pixels. A microscope has a very limited depth of field so doing it the way you suggest would mean 100s of exposures for each frame. You wouldnt need to take 700 exposures for each fram though since the whole chip wouldnt be visible, more like 30 exposures in the x direction and perhaps 5 (?) in the y direction. That would mean that the raw photos would need 100x30x5x60 exposures per second of animation or very roughly a million exposures per second of video. It would take a cluster of good computers and months to put it al together. Perhaps petabytes to store the raw data.

Interresting thought experiment but I gather Ive spent enough energy on this already... It is really an example of a sunken cost fallacy on a quite large scale. Edit: Thinking some more about it, it would be feasible to do a really close up shot, closer than closing scene the way you describe. The focus stacking would be the difficult part to pull of, but with automation, anything is possible. My calculation above is of course off by an order of magnitude.

Oh yikes, a lot of work indeed!  Great job.

To be clear, just imagining the possible; putting it into practice however, well, might be as you say.

Not sure what you mean by X/Y exposures?  I was thinking something like, CNC camera path, focus stacking (should be reasonably automatic? and nothing crazy, can keep some tilt-shift effect in the shot, 10-20 frames might be enough depending on viewing angle??), then move on to the next frame and so on; let it run, could save all the frames if you like but as long as the auto stacking is good enough, can save some hard drive there?

Well, maybe I'm severely underestimating just how narrow the depth of field is, and how many layers are needed for a proper high def scene.  It doesn't need to be extremely high resolution, would be enough to see the perspective effect, but more is always nice, too... depends how long you're willing to spend on it.

Cheers!

Tim

[mawyatt]

Very impressive work indeed

Really do appreciate the effort involved, as we haven't done any Stack & Stitch images beyond 30,000 X 20,000 pixels yet, and know what effort is involved in doing this!!

To get to this level we started back in ~2000 imaging the chips we designed, this was thru a microscope. Then moved to a setup based upon the Cognisys Stackshot System for focus stacking, then custom assemblies using reversed lens and then moved to infinitely corrected Mitutoyu types using custom tube lens assemblies.

About 10 years ago we began to develop our own core mechanical setups based upon THK KR20 rails with 400 step precision motors and pioneered the use of custom controllers for image stacking based upon Trinamic TMC controllers. These Trinamic controllers were some of the best stepper motor controllers available at the time, and likely still are. Soon we evolved to a custom completely automated 4 axis Stack & Stitch system to collect images for the massive pixel chip image processing effort. We even developed custom LED controllers for "strobe" type LED illumination use. Edit: We use Zerene for stacking and PTGui for stitching (our large S&S sessions choked PS).

About 5 years ago we ventured into the piezo electric area with nanometer levels of position resolution utilizing repurposed Physik Instrumente actuators and custom controllers. Since we haven't done a new chip design in the past few years, all this equipment has been packed up and sent to storage. If we do a new chip design, then we'll unpack and setup these Stack & Stitch systems and continue on.

Numerous folks/companies/universities have inquired about imaging their custom chips for a fee, but we've decided not to pursue this effort. Reason is the time consumed and massive effort involved just isn't something that's sustainable from a business standpoint, only a hobby can justify the efforts involved. This is why we have the utmost appreciation for SpiralElektronik and noobs efforts

If you want to follow some of our work with Chip Focus Stacking, Stack & Stitch Imaging, Trinamic Controllers, PI stages, and so on, see the Photomacrography site and search under mawyatt (user name).

https://www.photomacrography.net/index.htm

Here's an old, low resolution chip image you might appreciate, chip tweezers slipped

Have more but don't want to dilute the OP thread, unless it's OK.

Best,

[mawyatt]

Well, maybe I'm severely underestimating just how narrow the depth of field is, and how many layers are needed for a proper high def scene.  It doesn't need to be extremely high resolution, would be enough to see the perspective effect, but more is always nice, too... depends how long you're willing to spend on it.

Cheers!

Tim

Depth of Field at 50X with NA of 0.42 is ~3um, with Effective Aperture at sensor of ~60.

Best,

[amyk]

It's not a full die photo nor has the metal been stripped, but have you tried contacting anyone at the Visual6502 project? They might be interested in whatever images you do have.

most of the people that made that are now retired and the design files are stored in some archaic format that nobody can read anymore. The computer systems , hardware and design software are long gone.

Intel released the 4004 mask set many years ago:

https://www.4004.com/index.html

[mawyatt]

Here's an old link some might find useful. Be sure and visit the sites indicated, as there's quite a bit of information. We've been doing this for a couple decades now 

https://www.eevblog.com/forum/projects/diy-focus-stacking-for-macro-photography/

Best,

[SpiralElektronik]

Here's an old link some might find useful. Be sure and visit the sites indicated, as there's quite a bit of information. We've been doing this for a couple decades now 

https://www.eevblog.com/forum/projects/diy-focus-stacking-for-macro-photography/

Best,

I wish I had known about you guys before I started this journey. I have fallen into quite a lot of traps doing this project. Up until I saw your focus stacked image of the corner there, I hadn't really found an example of a "true to reality", real color die shot and thought I had a unique edge. But my research must have been affected by confirmation bias. Due to the questions I get here in this thread, finding your work etc I feel compelled to tell the story so you can understand my motivations.

Approximately four years ago I got hold of a quite nice wafer prober. Mitutoyo optics and a rock solid mechanical platform with motor control in xyz. It had been sitting around for years though so dust had creeped in and the grease had dried up and hardened.  I spent a year learning microscopy, regreasing the mechanics and optimising the optical path. I knew that my edge wasnt really the objectives, they are NIR optimised so they have a slight disadvantage in the blue part of the spectrum and they are long working distance meaning they have a less than optimal numerical aperture. The advantage I saw to the wafer prober was the mechanics, although old and manual they made it possible to capture a large number of exposures controllably. I set out to capture "the highest resolution die shot ever taken"(tm).

When I started taking the images I thouight that taking the images would be the hard part, stitching them together would be easy considering that there is a variety of panorama software out there and people are making gigapixels panoramas of landscapes all the time. I did a couple of captures before I got the optical path optimised enough, but they all suffered from heavy vignetting which gave the software a clue on how to stitch them which meant it seemed to work doing pseudo-automatic stitches of parts of the chip. But when I finally had optimised the optical path enough so that the vignetting wasnt a problem anymore, the software started making mistakes. Most automatic stitching software can do fairly correct stitching of parts of the image but it is nowhere near accurate or reliable enough to do a pixel perfect stitch of something this size. When I started realising this I hade already sunken several 100s of hours into the project. I thought that the images coming out of the project where amazing. But they always fell way short of what I wanted. I saw the potential but I also saw a lot of flaws. So I persisted. The problem is that you have to go through the whole process, from raw-format conversion through to stitching in order to see a final result. The cycle time is large from raw files to final images. And its not before you see the final image that you know for sure what is wrong and how it will be perceived. So you start over. And over. And over.

After trying  Microsoft ICE, PTgui, Affinity photo, Fiji(imagej) etc etc I ended up using Hugin in the end. It is free, open source and full featured. Once you have learned the slightly less than intuitive interface you have all the manual control you could ever need. And it is reliable enough to do large projects. But the automatic algorithms make way to many mistakes with this material. Also due to optical aberation, there is no perfect coordinate system. There is no way to _perfectly_ place all the images it is a slight compromise even in the best of circumstances. Also the image is way to large to process all at once. I divided the image into approximately 30 parts, stitched them in separate projects and then stitched the 30 parts into four parts due to tiff-file format limitations. Then I used photoshop to stitch the four parts into the final original.  All of the parts have to be perfect since a small error in one of them means that there will be a mismatch visible in some other part of the image since angular errors amplify with distance. I must have gone through the process of raw file conversion, stitching parts, and then stitching the final image 20-40 times before I got a result that could be considered as close to flawless as I could make it. The repeating features on a lot of parts of the CPU means that what you are looking for to set control points are not features of the CPU but specks of dust, scratches etc in a lot of parts. And you have to set them with pixel accuracy. The software can do statistics on how far off a control point is but due to optical errors that statistics will never go to zero error no matter what you do.

Ive discussed at length with people online if I could have done something differently but as far as I know the best way to avoid all that  manual labour would be to keep track of the exact position of each exposure while taking them with encoders. That is the strategy I would chose if I would do this again. I really think that if I can get the energy and the funds, I could eliminate the enitire manual process described above. That would turn the cycle time from  years down to a week of the microscope doing the work for me. And getting there would take less time than doing this whole process again with the experience I have.

So its really a sunken cost fallacy. Once I had gotten deep enough into this I really didn't want to give up. The images I was making was really cool but not cool enough to be considered a really nice photo. So I kept trying. By the time the main photo was created I was totally spent in all sorts of ways. Unfortunately this has meant that the video (probably my main end goal) only got a couple of weeks or perhaps a month of work, with me learning blender and finding a method in blender as I went along. I spent approximately two days making some music and that was it. The video is therefore lacking in storytelling and not really as  interresting as it should be. However it is the best I can do under the circumstances. And I have yet to see an example of a photographic deep zoom similar to this. Ofcourse someone has made it already somewhere but I haven't seen it.

So yeah there you have it, three years of work, 800 pseudoreluctant views, some criticisms and a few prints sold thats it. Now I have to find a way to recover. Im posting this here because I need to learn my lesson, getting for instance Mawyatts perspective helps me move forward and analyze what I did wrong.

My motivations where not to reverse engineer, I am an electrical engineer by trade but reverse engineering isnt something I would do for fun. I wanted to capture what the silicon looked like in a way that would show the overwhelming complexity of silicon. Something I actually think I succeded in doing. The image looks stunning when printed in a large format. Because of the high resolution, it would be possible to print it in 4 by 6 meters and still maintain 300 DPI. Here is a photo of a 1 meter wide print:

[mawyatt]

Thanks for the details, and we know exactly what are talking about

In our case we had no references, so had to create things on the fly. The motivation for us was the display of the chips we had designed and use these for presentations, customers and such. I remember showing some images on my MacBook Pro to the CTO at IBM during a break at a meeting way back in ~2011, he called everyone over to take a look. Later many of these were printed and displayed in the executive conference room. Keysight, DARPA and many others also have images displayed at various sites with the techniques we developed.

It does get expensive, and probably invested ~$40K or more out of pocket, and many tens of thousands of hours to boot!! This is exactly why we never offered anything for sale, too much effort for a little $ return.

Speaking of effort, when we ventured into the massive S&S area after a couple chip images the effort was overwhelming, many weeks of image capture followed by all the post work for a single image!! This precipitated into developing a fully automated S&S system which could take 4~8 hours continuous operation, collecting maybe 20~30,000 individual images for post processing. This effort destroyed a few strobes (overheated) and we had to develop/use different strobes/lighting techniques, and wore out the shutter on some camera bodies, so we moved to electronic shutters. We've stuck with Nikon cameras from the beginning way back in ~2000, still have a D800, D800E, D500, D850 & D7 at the present.

Stitching brings in another set of issues, folks that do this with insects and such, don't have to deal with the ultra-precise alignments of chips. We quickly learned the value of tele-centric lens and design. We have a few exotic lens/designs that we used, all very expensive, and many are not available today except on the surplus market. We even have an old Nikkor PN105A F2.8 lens that was designed and used to make copies of movie theater 35mm film frame by frame from the film masters. This lens can out resolve the D850 at 1X, and has almost prefect edge to edge performance, in fact center and edge resolution are the same with no apparent fall off on a full frame 35mm sensor!!

So we have the utmost appreciation for what you've gone thru, and also what lies ahead.....read "what's in your wallet"

And, BTW your work is superb, please it keep up

Best,

[SpiralElektronik]

So we have the utmost appreciation for what you've gone thru, and also what lies ahead.....read "what's in your wallet"
Best,

Thanks, but im not sure thats funny...

[mawyatt]

Better lenses, better cameras, better fixtures, pixel shifting, nanometer resolutions........

https://www.closeuphotography.com

As you asymptotically approach that perfect chip image with infinite resolution 

Best,

[magic]

For the record, I have had good luck stitching a few IC mossaics with tens of frames fully automatically in Hugin, after messing with configuration a little.
That being said, my chips were merely a few mm² (and image quality wasn't quite the same too, but that's another matter )
https://www.eevblog.com/forum/projects/decapping-and-chip-documentation-howto/msg3613732/#msg3613732

[SpiralElektronik]

Yeah tens of frames are no problem, thats what lead me astray. But i think youll find you get better results doing it manually once you get the steam up. I don't use the automatic features even for very small projects like this  U776HC opamp. 6 frames using a 10x objective iirc.

[magic]

I'm too lazy for that

CPFind gets majority of the points bang on, provided that I restrict it to searching correct image pairs and only compensating for camera movements. The points it gets wrong are typically a few px off and don't cause significant misalignment (their error contributions likely average out anyway), or they could be easily corrected manually if problems arise. That's still much less work than doing everything from scratch. It's a nice thing about Hugin that it enables mixing manual work with automation in quite arbitrary order and proportions.

[mawyatt]

When you do significant depth images like we have done in the past, with multiple stitches, things become much more difficult. This is where simple lens and techniques fail due to image perspective shifts, and for good results one needs to look for more tele-centric lenses (at higher magnifications some typical lens aren't too bad tho, but requires trial and error).

As mentioned by OP SpiralElektronik, larger chips like the 486 are quite difficult, especially with depth (tilt) and require significant skill, technique and patience than smaller planar subjects, and are generally well beyond the typical hobby level efforts!!

We tried Hugin long ago, after PS failed to stitch larger image sessions, and another program (can't remember the name), but ended up using PTGui as it was the only program that actually worked on all our larger stitches. PTGui also requires significant learning curve, and much hand holding to successfully render a quality image, but can do so with proper use.

Honestly, we didn't give Hugin as much attention as we should since Rik Littlefield (author of Zerene stacking program) recommend PTGui and that's what we went with. Never tried CPFind tho, it might not have been available when we started the massive S&S efforts many years ago.

Best,

[mawyatt]

Yeah tens of frames are no problem, thats what lead me astray. But i think youll find you get better results doing it manually once you get the steam up. I don't use the automatic features even for very small projects like this  U776HC opamp. 6 frames using a 10x objective iirc.

Agree, these smaller planar subjects with a few simple stitches are not that difficult and don't require much from the setup and lenses involved. However, even these subjects can and will cause issues when they are positioned for depth rendering.

When one wants to print depth chip subjects the size of walls rather than a postcard on a screen for the net, and still retain resolution so one can "zoom" with their feet, this is where the skill, knowledge and effort go from Elementary to Post Doctoral levels. When you mentioned you had acquired a probe station and using Mitutoyo lenses, we knew where you were heading

Best,

[magic]

Honestly, we didn't give Hugin as much attention as we should since Rik Littlefield (author of Zerene stacking program) recommend PTGui and that's what we went with. Never tried CPFind tho, it might not have been available when we started the massive S&S efforts many years ago.

Hugin is still no walk in the park, one needs to know how it works (not too complex but RTFM is in order) and what to click and when in order to make progress. The default settings and the "beginner mode" are hopeless - they try to match every image with each other, wasting a lot of time and usually getting confused by similar features which just aren't the same.

The upside is that everything produced automatically can be reviewed and corrected, and most of the tiresome work can be automated. There are still rough corners, like it's fairly unobvious how to coarsely pre-align the images on a grid to improve automatic alignment.

Good for the price, though