Category: Uncategorized

In the world of electronics assembly, parts are really small, quantities are really large, and equipment is really expensive. The process of putting all the components onto circuit boards thousands of times per hour with really high accuracy is challenging, but tools are getting easier and cheaper every day.

At my job (Quietyme), we’re doing our own circuit board assembly. We had to purchase some equipment, like a pick and place, which for $5000 automates the process of taking the small components off of reels and putting the components on the circuit board. We also have a stencil screenprinter and a reflow oven (toaster oven). Total investment so far has been under $6000.

The most recent tool to add to our arsenal is an Automated Optical Inspector. The purpose of the AOI is to examine the circuit board to find defects in the assembly process. Sometimes a component doesn’t solder on correctly, or there is a bridge where there shouldn’t be. These need to be found and fixed before they are programmed and used. We had been doing it manually with just visual inspection, but that was tedious and not very accurate, especially after the 100th board.

I had been reading into OpenCV and decided I’d give it a shot. The idea was to take a regular webcam, point it at the circuit board, do some image analysis to figure out if there were parts that didn’t match a “good” board, and highlight those so that a full inspection by a human wouldn’t be necessary. After a little bit of playing, I realized it could work. I designed a holder and laser cut it out of acrylic, then glued it together. It was pretty slick. With the threaded rod I could raise the camera up until the PCB occupied the entire field of view, so no pixels were wasted. I wanted consistent even bright light, so I added a bunch of white LED strips. I immediately noticed a problem with hot spots, so I added a layer of acrylic with a diffuser (a piece of paper glued to it).  Finally, I glued the bottom of the enclosure to the base. The PCB was designed for the enclosure, so it was a perfect holder that would consistently keep it in the same place.

The camera is a 720p Microsoft LifeCam that I had laying around. In the next version I’ll upgrade to a better camera. The LED strips are just 12V white LEDs.

Now the software. I used the OpenCV CV2 python library, and ran it both on my main dev computer (Linux Mint), and my test laptop (Win XP (shut up)). The older computer was definitely slower, and setting up OpenCV in windows is a pain, but I made it work.

The first step is to get the video stream from the camera and display it. Fairly simple. Then I experimented with diffing against a known good board. The idea was that by subtracting the video from a good image, only the differences would be visible. This is good in theory, but it didn’t work at all in practice:

This is way too busy to think about and try to analyze to find poorly placed components. It wasn’t going to work.

The next idea was to mask out anything we didn’t care about. There are unpopulated components on this board, and we don’t care about the silkscreen or the outside edges. There’s no point showing them to the user. So I created a black and white mask:

 

This mask is displayed over the top of the video stream, so that the operator only has to look at the unmasked parts to evaluate the board. Like so (note that this is an intentionally sabotaged PCB which has bridges, tombstones, and misaligned components that I put in to test the pattern matching):

Great! Now the work is already a lot simpler. From here I wrote a quick routine that would zoom in on the Zigbee chip to digitally and show a 4x zoomed chip to look for bridges. So no pattern matching yet and we’ve already significantly improved the inspection process and reduced eye strain.

The next step was adding in the pattern matching. The idea is to use the OpenCV template matching feature to identify components that are correctly soldered and use those as a template, then when the video stream sees those templates matched it’ll draw a black box over the component indicating that it doesn’t need to be examined. So the above image after pattern matching should look like this:

Bam! Now we’ve gone from having to inspect a full board to just having to check over a few components! In reality it ends up not matching perfectly so we usually see a few extra components that are just fine, but it’s better than having the threshold too low and accidentally blocking out bad joints.

So how does the setup process work? By drawing boxes! In the python application, just drag a box around the component you want to use as a good template, and it will create an image and store it and use it as a template from then on. I started by first looking over the entire video stream for each template, with the idea that a capacitor in one place might look like a capacitor somewhere else. This was a dumb idea and slowed down the system to unusable. Then I got the idea to save the image coordinates as the filename, and only look within a certain bit of slop within that area for that template. That got me back up to near real time processing of the feed. It also allowed me to have multiple good templates for a single component to bump up the likelihood of matching without sacrificing the threshold of matching.  So it ends up working pretty well. Here are some examples of component templates:

 

So to summarize, I’ve put a few hours of time (<20), a webcam, and some hardware into this, and I’ve got a functional AOI to add to our assembly line. It’s really only useful for small circuit boards, and a single camera that doesn’t move definitely has limitations, but it’s really a testament to the awesomeness of OpenCV for making this so easy.

It should be noted that a cheap AOI can be had for $20k, and the ones used in some factories sell for over $100k. Mine is by no means competitive with theirs, and couldn’t find nearly as many types of faults on as large a board. They move the camera around and look at solder quality using colored lights and blah blah blah. But for the price of a webcam, some acrylic, an LED strip, and a few hours of time, this is a pretty big step.

Want the code? Here ya go: AOI Python code

Yep, I haven’t updated in a while. So what happened in the last year?

• Went to china to participate in HAXLR8R, the first hardware startup accelerator, which took place in Shenzhen. Read about it here!
• Came back a few weeks early because I couldn’t get what I needed there, went to San Francisco for demo day and launched a kickstarter project.
• Demo day was a flop and my kickstarter failed. I just wasn’t ready for it. I didn’t have a team, I was exhausted after China, and I didn’t know how to do media campaigns.
• I learned my lesson and brought on a couple of co-founders to handle marketing and business development.
• I learned another lesson about hiring people I’m not completely comfortable with, and was able to fire him before the paperwork was signed making him official.
• I drove across the country with my girlfriend to go to an amazing wedding and see friends and be with Erin’s family.
• I continued product development for a long time, making it manufacturable, talking to U.S. factories, jumping through regulatory hoops, getting prototypes in the field with beta testers, and in general making the product and the business awesome.
• I sent balloons into near space.
• I have become an expert in many things business and startup related.

And now you’re caught up. 🙂

A couple months ago I applied for a program in its first year for hardware startup companies. Called HAXLR8R, the goal of the program is to make manufacturing in China more accessible to startups by providing workspace and tools and mentors and experts to ten teams for three months in Shenzhen. After those three months, the participants spend a week in San Francisco and give a demo to investors.

I found out recently that I was accepted for my project on the Portable Electronic Scoreboard. I’ve been working on this project for a while, and now is a good time to go to China because I need to take it to production, and this is a great way to do it.

Over the next few months I’ll be updating a new blog that I created called Engineer In Shenzhen, which will have personal posts about what I’m going through, and advice and articles for other people who are interested in the process and what it takes to work in China and outsource manufacturing.

If you want a postcard, make sure I have a current address for you. I’ll try to keep everyone updated with progress as I can.

It occurred to me that a lot of people may be out of the loop with what has been and will be happening over the next while. Here’s a short description of the plan:

Erin is going to graduate school in Madison, WI, and I am moving with her.

On July 31 our apartment lease in Richland expired.  On August 15 our apartment lease in Madison begins. For the two weeks in between, we are house-sitting for a friend and storing our stuff in their garage. On August 15 we pack everything up again and begin our 3 day trek across the nation, putting us in Madison on the 17th. We’ll live there for a couple years while she gets a Masters in GIS (Geographic Information Systems).

In anticipation of this move and change in lifestyle, I’ve made quite a few changes already. A few days ago I sold my car; its useful life had was ending and was getting to a point where it would need more and more maintenance and repairs. Since I work from home, and Erin will be taking public transportation in Madison, and parking is expensive, it made sense to only keep her car. In January, I left my job at PNNL to start my own business. I was subcontracting back to the lab for a few months after that to retain some stable income as I built up my own business. I am currently freelancing a little (and open to new work if you have leads), but also working on the portable electronic scoreboard and an automated dog-sitting application. When we move I’ll continue working on them; in fact changing my location doesn’t change a thing about how I do my work other than that my office will soon have better windows.

The hazop chapter is drawing to a close and the next adventure begins shortly.

It’s been four months since I started on my own. The first weeks were a whirlwind of applications and registrations, some of which are still in process. I’ve been building up projects, finding my niche, and getting used to and developing my processes.

In the beginning I started by reading a lot of books; everything from business law to marketing to general advice on working from home. I feel like I’m getting an MBA by fire, but the stakes are a lot higher than being in school.

Here’s an overview of what I’ve done in the last four months:

• Registered my business with the WA Secretary of State, the WA Department of Licensing, and filed for an EIN number from the IRS.
• Applied for a trademark on the WYZGYZ name. I still haven’t heard back about that, but the process takes some time.
• Set up a business bank account.
• Set up http://wyzgyz.com.
  
• Met with a lawyer
• Met with an accountant

On the other end of the business, I’ve been involved with quite a few projects:

• Two contracts back to PNNL to continue to work on projects I was doing when I left. This is my source of income while my other projects are still in development.
• Volunteer development for the Atomic City Roller Girls. I redesigned their site and have been helping them with posting and using it.
• Working on a project called the electronic scoreboard. I built a second prototype, set up a web site, and am currently looking for manufacturers and getting feedback from users.
  
• And a few more things I’m not ready to talk about yet.

Between learning about and getting used to owning a business, my own projects, and my contract work, I’ve been pretty busy with WYZGYZ. I’m not discouraged, though I have had a couple rough days and some tough decisions to make. Rather, I’m excited that I’m doing this, and I have high hopes for what it will become.

It has been long overdue, but I’ve finally taken a giant step in my career. 1/11/11 was my last day working at Pacific Northwest National Laboratory. I have left to start my own business, and I’m very excited about it. The last few days have been a flurry of paperwork, but I’m making progress and getting to a point where I can finally start talking about it. Assuming all my paperwork goes through in the next couple days, I’ll officially own my startup: WYZGYZ (pronounced ‘wise guys’).

I’ve always had a love/hate relationship with PNNL when I was there. I really appreciated the opportunity to start working on challenging projects and contributing to publications and leading-edge research right out of university. I have many conference and journal papers and presentations, and even a few patent applications through the lab. I’ve gotten the chance to work with influential people at the top of their careers, and travel around the United States to meet and work with dozens of others on some pretty fantastic projects. I’ve learned a lot and become a pretty good software developer. For my first job out of school, the lab was perfect for me.

But there were things that bothered me about the lab; things which ultimately made me decide to leave. First among them was my inability to pursue my personal ideas. It’s ok to own my brain when I’m being paid for it, but there were things I wanted to do that just couldn’t be done while I was an employee at the lab. Second was bureaucracy. Working in a government lab was frustratingly slow, and many times I saw my ideas ground down and starved because they couldn’t get traction or took too long to get funding. The red tape machine is formidable and to me seemed counterproductive to innovation and progress. Finally, there was impact. Over the course of my tenure at the lab, I wrote a lot of software. Much of it was research software that was used for a prototype or a demo, shown to the client, and archived. In the latter half of my tenure I worked on software that did get used, sometimes in some pretty exciting ways, but never beyond a couple hundred users. It seemed like I would never be able to develop a product that could get through all the barriers to find use in the general community and have a meaningful impact on thousands or millions of people. Publications required approvals, and even posting to bulletin boards was regarded with skepticism. For a national resource, it seemed too closed-off.

WYZGYZ is my shot at building a business that has the things I liked about the lab and addresses the things I didn’t like.We’ve also been sponsored by Carlson Knives, so check his products and let us know what you think, .

My company will have three parts: contracting, products, and research. Contracting will allow me to work for clients and generate stable income. Products will allow me to develop my ideas into products that I can market and sell. Research will allow me to explore ideas and either turn them into products or publish them to the community.

Everything is in the very beginning stages, but I have a plan, I have all the resources I need, and I have a lot of hope and motivation and skill to make this work.

This is a shining example of a highly productive Sunday afternoon. I separated the marshmallows out of the cereal. Then I threw away the cereal. Then I ate the marshmallows. That’s a normal sheet of paper to get a sense of scale.

For what it’s worth, the cereal was stale and wasn’t going to get eaten anyway.

This weekend I was busy. Saturday morning we went to the raceway in West Richland to watch some friends autocross. They weren’t scheduled to go until later, though, so we stayed for about an hour, then headed to another engagement; the Tri-City Wing Wars.

The Wing Wars was a competition among local restaurants to see which one had the best wing. For $5, anyone could enter and have access to unlimited wings from any of the stations, all of which hid their logos and their brand name. People could vote for their favorites, which would be revealed at the end.

In reality, more people showed up than they expected, and they ran out of wings after about an hour. Sadly, I was one of the reasons they ran out. I managed to consume two dozen chicken appendages in that hour, though to be fair I didn’t target my consumption at any particular vendor, though the Parmesan wings were my favorite. I saw a few friends there, but after the wings ran out, so did we, heading back to the track to see if we could see the friends race.

We arrived in time, and Ricky offered to let me ride with him. He has a nice convertible, so I was happy to accept. The helmet, though tight, wasn’t the claustrophobic experience I expected, and having the top down held the heat and gas smell at bay, so I was quite comfortable. The racing itself was a lot of fun, and I felt like the entire time we were pushing the car to the very edge of its capabilities, and that if Ricky pushed it any more or less expertly, we would certainly spin out. It felt like we hit every cone, but in fact he had a great race and only hit two.

Later, Naomi raced with Ricky, which was fun to watch, and we could all hear her screaming (we assume with joy) occasionally.

Finally, Dan was up, and he let Erin ride as his passenger. She put on her helmet and joined him in the car, a very sporty little thing into which Dan had obviously put a lot of care. I climbed the lookout tower and caught the ride on video; it turned out to be a great run, netting him first in his class for the day.

After the autocrossing, we went to our friend Dimple’s house for pizza and a movie, then went back home.

Sunday morning, I was up early. I had planned to go hiking with Jim and Erin that day, but Erin bailed at the last minute, citing work. Jim and I still went, and it took about an hour to get to our destination; the white bluffs of Wahluke wildlife reserve. We chose a path that wasn’t exactly the famous white bluffs but less traveled. Half of our walk was on an old blocked off road, and the other half was along coyote trails that were barely visible. Along the way we could see the whole Hanford area.

Along the way was an old earth mover. The handle wasn’t attached to the door, but it was easy to insert and open. The key was lying on the cab floor. The battery, however, was dead. Still, it was good for a photo.

I took a few panos on the hike, too.

My biggest lesson from U.S. History class in High School didn’t have anything to do with History. It was a lesson that’s stuck with me ever since, and it has affected me in many aspects of my life. Getting things done is really intimidating, until you start.

Within the first month we had our initial assignment, but this was an AP class, not a regular history class. Our task was to write three essays, and we had a week to do it. I went through so many emotions; anger, outrage, frustration, hopelessness. I started to do the work but it was impossibly daunting. There was so much that I didn’t know, so much research to be done. Each question could have been a masters thesis and seemed to require citation of dozens of materials. I remember crying to my dad. He couldn’t do anything for me, though. Ultimately, I got it done, and on time, by just getting started.

My problem, and my major block, was knowing that I didn’t know enough. There was no way I could read all that I needed to to make a cohesive and complete argument. I had to give that up. The trick was to start with a decision already, start writing, then look for supporting facts in my research materials. I didn’t need to find the right answer; I needed to be able to defend my answer. This worked most of the time, because our lectures were usually on the topic of the essays and we had a good idea, but it didn’t always happen that way. Sometimes I would start with a thesis, but in the process discover that it was completely wrong. Since I had already started, and was already in the process, it was easier to go back and modify than it would have been to start over. Eventually I would get to the right answer anyway.

At the beginning of the school year, it took me the whole duration to write the three essays, which were assigned every week, but almost all of that time was just trying to get started. By the end of the year, and for the AP test, I was producing all three essays in under an hour, at roughly 2-3 pages each. Sure, I had gotten better at researching, but I’d also become less intimidated by the assignments.

That’s been a lesson for many things. With skydiving, so much worry and preparation goes into that first jump, but after getting used to going through the motions, it becomes routine, and what seems like an impossible feat to those who haven’t done it yet is just another day for someone else.

The truth is everything is intimidating until you start doing it. Then you learn a lot really fast. Then you become good at it. If we all just accepted that once we started doing something it would be fine, and skipped the intimidation step, life would be a lot easier for everyone.

It’s been over a year since I’ve posted, which is not cool. I’d like to say it was the fault of the technology I was using, and that it was a hurdle, but that’s just making excuses. A lot has happened in the last year. The girlfriend and I have traveled to Vietnam, Thailand, Vancouver, Belize, Guatemala, Seattle, Montana, and a bunch of other places. Work hasn’t been much different, but it’s always interesting. In fact, a lot has happened, but over the course of a year, it condenses quickly into only a couple sentences. This is no longer acceptable, so I’ve decided to change technologies, and devote more time to posting about the things that I do. We’ll see if it works out well.