A few years ago on a camping trip my friends and I started playing around with drawing shapes with our headlamp while someone else took a 30-second exposure with the camera. I drew Pac-Man chasing a ghost and it came out fairly well for blindly waving a flashlight around in the dark.
I thought it might be cool to build a robot that could draw a picture like this. I built a robot that moves a LED around a grid and then turns on the LED to the desired color at each location on the grid. I used my DLSR to take a very long exposure while the robot is moving the LED around the grid. The end result is a photo of the LED at all of the various points in the grid which (hopefully) looks like the image the robot is trying to reproduce.
LED Photos
Pac-Man
Given the photo that gave me the idea for the robot I had to try a Pac-Man image first. This was a nice photo to start out with because there was a lot of black in it so it only took the robot about 10 minutes to draw it. That made it nice for troubleshooting because I only had to wait 10 minutes to see if it worked.
The Starry Night
Next on the list was The Starry Night. This was a big jump in complexity over the pac-man photo. It took me a few attempts to get all of the kinks worked out. There aren't many all black pixels in this one so it took about 30 minutes to complete. It is a long shot from a perfect replica but at least you can tell what it is :)
Lego Mona Lisa
I stumbled across this image and it seemed too fitting for this project :)
In Action
How It Works
The LED
The color sensor from lego can also produce light of various colors but it is pretty limited. You can only tell it basic colors like "red", "green", "yellow", etc. I needed a way to produce a larger range of colors so I bought a dLight from Dexter Industries. It comes with four LEDs but I only ended up using one of them. Most of the time you see these used in robot cars so people can add turn signals and/or headlights.
The LED was a little bright though so I punched a small hole in a piece of black construction paper and taped that over the LED. That gave me a smaller light source which meant that I could squeeze in more pixels.
Robot Hardware
I needed the position of the LED to be very accurate so that I could draw the right color pixel in the exact spot where I needed it. I've tried doing super accurate movements like this before with a robot that drives around on rubber wheels but at some point the wheels always slip a little and that throws everything off. To get the accuracy I needed I built a platform with two rows of gear-racks on top. The LED sits on top of a motor which has gears for wheels, these gear wheels drive along the gear racks. This is how the LED moves up and down.
Getting the left to right movement was a little trickier. I put wheels at the ends of my "up/down" platform and attached a long arm of beams to the platform. I put gear-racks on top of the really long arm and used a 2nd motor to turn a gear that joined with those gear racks. This gave me a way to slide the "up/down" platform left and right.
Robot Software
I wrote small program in PERL reads an image file and gets the color for each pixel in the image. This ends up being information like "Pixel 10x50 has Red=57, Green=140, Blue=200". I just printed all the information about what color each pixel should be into a simple text file. I then copied that text file over to the robot where the robot would read the text file to learn how much Red, Green, and Blue for the LED to output at each location. Then you just have it move to every location listed in the text file and tune the LED to the correct color.
I posted a video on youtube of a Lego Mindstorms NXT robot of mine, playing Drop7 in sequence mode to a score of 5,205,955. Some of my friends told me I should blog about it to explain how I did it, so here is a rather lengthy explanation of everything that went into the video. The robot plays to the end of Level 71, it clears the board 23 times, the longest chain is x29 and is worth 367,087 points. There are also chains of x26, x25, x24, two x23s, two x22s, etc, etc.
The Inspiration
I started playing Drop7 on my phone a few months ago and eventually stumbled across a video of someone who scored 5,000,000 points on Drop7 in sequence mode. For those of you that have never played Drop7 is it sort of like a math based version of Tetris. You have a 7x7 grid where you select which column to drop a disc in. You earn points by making discs explode, preferably in a long chain which earns more points. Making a disc explode is easy, if you place a disc with a 4 in a column with 4 discs, all of the 4s in that column explode. The same rule apples to having 4 discs in a row, when that happens all of the 4s in the row explode.
Anyway, I started thinking about how the person that made the video figured out what sequence to play the discs in to get such a high score. They were playing in sequence mode which means that the same discs are played every time you play the game. I was sure they wrote a program to figure out how to get such a high score and I like to do geeky little projects so I wrote my own program that simulates Drop7. I figured I would have it run through a few million permutations that one could play the discs in and see which permutation produced the highest score.
Permutations
Just how many permutations can the discs be played in? Let's say you only know what the first four discs will be in sequence mode. There are 7 different columns you can drop each disc in so the permutations will be as follows (the numbers represent which column to place a disc in):
Count
Permutation
#1
1 1 1 1
#2
1 1 1 2
#3
1 1 1 3
. . . .
#2399
7 7 7 5
#2400
7 7 7 6
#2401
7 7 7 7
This comes out to 7^4 or 2401 permutations. OK 2401 isn't such a huge number for a computer but let's expand this and assume you know what the first 100 discs are in sequence mode. 7^100 is a GIGANTIC number of permutations, 3.2344765096247579913446477691002e+84 to be exact. That number is so huge it is hard to wrap your head around it. To put it in perspective if you stacked up 3.2344765096247579913446477691002e+84 sheets of printer paper the stack of paper would be about 3.29 light years tall! Needless to say I don't have access to enough computing power to crunch through that many permutations of Drop7 within the next few billion years.
I decided to break the first 100 discs down into groups of 8. I could run through every permutation that one could play the first 8 discs in, find the permutation that resulted in the best score and then move on to the next batch of 8 discs, etc. This wouldn't give the absolute best possible score one could get out of the first 100 discs but it would spit out a sequence to play the discs in that would produce a very high score.
Drop7 Simulator
At this point the little simulator program I wrote was in a language called PERL which is an easy language to program with but is slow compared to a complied language like C. It took my PERL program quite a while (days) to crunch through the first 100 discs in groups of 8 so I rewrote my simulator in C which is much much faster, in this case the C version was 74x faster than the PERL version!! I then moved from analyzing the discs in groups of 8 to groups of 12.
I was doing all of this on my home PC which has four 2.8Ghz cores. Crunching through the discs in groups of 12 took a long time, each group of 12 was 13.8 billion permutations (7^12) to simulate. I knew I wouldn't be able to do move up to groups of 13 on my home machine, not unless I was willing to let it run for months. The larger the groups of discs you analyze the higher scoring sequences the program is able to find so I wanted to move from 12 to 13 if not higher. I was chatting about this little project with a co-worker and he gave me access to one of his machines at work that has 32 cores. It is a Linux machine running on a Cisco UCS B200-M3 to be exact. It isn't often that one maxes out 32 cores for days on end...here is a screen capture that shows them all pegged at 100% :)
In the end I was able to crunch through the discs in groups of 14, each group of 14 has 678 billion permutations. I knew what the first 680 discs were in sequence mode which means there were ~49 groups of 14 discs to analyze. So 678 billion permutations per group x 49 groups x 14 discs per group = 465 TRILLION discs were "dropped" in my Drop7 simulator. Even with 32 cores running 24 hours a day it took weeks for it to produce the sequence I used for the video.
I posted all of my code for the Drop7 simulator on github. If anyone wants to tweak it to find a better Drop7 algorithm you have my permission to use my code. https://github.com/dwalton76/Drop7-Sequence-Mode
Sequence Mode Discs
I found a post on a forum somewhere that listed all of the discs that would come in sequence mode for the first 10 levels. To figure out the future disc beyond level 10 I would play to level 10, then record a video of myself playing, then watch the video to figure out what the new discs were. The only problem with this approach was that I had to keep playing from the beginning of the game up to Level 10, or 11, or whatever level I was stuck on at the time, in order to figure out what the next discs were. This was taking a while and either my wife or I jokingly said that we should build a Lego robot to play it for us. I'll go into the robot more in second but here is a table of all discs (a disc with a "?" means it is a solid disc) for the first 71 levels of sequence mode. Should someone else ever decide to do a project like this they shouldn't have to analyze game film of themselves playing Drop7....that part was pretty boring.
Lvl
Level Up Discs
One-By-One Discs
1
6?
4?
5?
7?
5?
1?
3?
2
1
1
6?
4
5
5?
5?
1?
6
1
5
7
5
3
6
3
2?
6
4?
2
5
7
1
2?
5
1?
3?
2?
4
2
1?
3?
6?
3?
3?
4?
4?
5
5
2?
1?
6?
2?
3
2
3
2?
3?
6
2
7
7
6
3?
2?
6?
4
7
4
5
4
3
6
1
6?
1
3
6?
4?
5?
7?
3?
3?
4?
5
7?
4
6
3
5?
7
4
3
5
2
6?
2
7
2
1
4
7
4?
4
2
7
3
5?
5
5
1?
7?
4
2?
2?
7?
1?
7?
1?
5?
7
7?
2
7?
2
3?
2
3
2?
2
6?
5?
4?
6?
5
7?
2
6
7?
7?
5?
4
2
3?
7
2
3
5
2?
3?
7?
1?
5?
4?
1?
2?
1
3?
4
1
3?
1?
1
6
1?
5
7
5?
2
5
5
6?
3
5?
4?
5
7
4
1
3?
5
6
5?
2?
3?
6?
2?
7?
1?
1
6
4
7?
3
7
3
7
7?
4
4?
1
5
1?
5
1
4?
4
3?
4
7
4
3
1
4?
7
3?
6?
7?
1?
3?
7?
3?
5
5?
7?
5?
5?
1
3
6?
2?
2
3
7
5
4
5
6
1
6
7?
1
1
2?
7
3
8
1?
5?
4?
1?
6?
3?
3?
7
2
2?
7?
7
4?
6?
5
7
5
7
6?
7
4
6
5
1
6?
3
4
7?
7
5
9
3?
7?
3?
1?
1?
1?
4?
2
6
1
3
7?
6
4
7
3
7
6?
5
5?
6
2
4
3
3
5?
7
4?
7
10
2?
1?
6?
3?
2?
3?
7?
1?
5?
4
3?
6
5
1?
7?
7
1
5
6
1
7
4?
6
3
6
1
1
1?
11
5?
3?
4?
5?
4?
5?
3?
5
6
4
7
7
4
4?
2?
5?
3?
4?
4?
4?
3?
4
3?
3
2?
1
4?
12
6?
1?
6?
6?
7?
3?
3?
7
2
7
5?
3?
6
3?
2?
4
6
2
2
5
7
1
7
5?
3
7
13
6?
4?
2?
5?
7?
7?
5?
1
4
3?
2
2
7?
3
2
4
2?
6
7
1
3
4
6
2?
4?
14
7?
7?
4?
2?
5?
1?
1?
3?
2?
2
7
5?
5
6
7?
1?
1?
4
3?
4
3?
1
6
5?
15
5?
1?
3?
3?
5?
3?
5?
2
2?
5?
3
1?
7
2
2
1
1
7
7
7?
1
4
4
16
4?
7?
4?
5?
7?
6?
7?
1
1?
4?
4
4
4
4
1
4
5
7?
2
2
4?
7
17
1?
4?
3?
3?
5?
2?
3?
4
5
3
2
3?
7
1
4?
7?
7?
3
2?
6
2
18
1?
5?
1?
3?
6?
4?
3?
3?
2
2
5?
6?
4
6
2
7
3?
7
6
1
19
1?
5?
7?
4?
1?
7?
6?
2?
3
5
4
6
5?
6?
1
3
4
1
1
20
7?
3?
6?
7?
1?
6?
2?
1?
1
3
6
4?
1?
1?
6
7
7
3?
21
3?
2?
1?
2?
5?
5?
6?
5
3
3
4
6
3
4
2?
1
1?
22
6?
4?
2?
2?
5?
5?
2?
4
5?
3?
6
6
5?
6
2
4
23
6?
2?
7?
3?
7?
3?
2?
1
7
7
1
5
2?
2
7?
24
2?
3?
6?
6?
4?
6?
1?
5
3
2
3?
2
1
2
25
4?
5?
4?
7?
5?
6?
2?
1?
5?
6?
3
5
6?
26
4?
4?
6?
6?
2?
5?
5?
2
7?
6
1
5?
27
1?
3?
3?
5?
4?
4?
7?
4?
4
3
7
1
28
3?
4?
5?
2?
2?
3?
3?
2
2
7
7
3
29
2?
5?
1?
4?
5?
5?
6?
6
2
1
6?
7
30
3?
3?
4?
6?
1?
2?
4?
2
3
5?
1
4
31
2?
5?
1?
2?
6?
2?
5?
4
7
3
1
1
32
5?
1?
1?
5?
4?
2?
3?
7
1
4?
4
5
33
7?
2?
6?
4?
7?
3?
3?
6?
4
4?
3
4
34
6?
3?
7?
2?
5?
6?
3?
4
6
1
7
2?
35
5?
4?
1?
6?
7?
4?
6?
5
2
5?
5
1
36
4?
6?
3?
7?
3?
3?
5?
1
5
1?
6
3
37
4?
7?
7?
6?
3?
7?
7?
6
7
3
7
4
38
1?
5?
1?
1?
6?
5?
5?
5
2
2
4?
7
39
2?
2?
6?
4?
2?
1?
2?
4
4
4
4
5?
40
1?
6?
5?
4?
1?
2?
5?
6
5
3
4
6?
41
1?
1?
2?
2?
6?
1?
1?
1?
6
6
7?
2
42
7?
4?
3?
7?
3?
3?
2?
7?
6
1
3
4?
43
2?
6?
2?
6?
5?
2?
4?
2
7
7?
4?
6
44
6?
4?
6?
3?
4?
4?
3?
5
3
6
3?
5?
45
7?
2?
3?
4?
5?
4?
6?
1
4
4
7
2
46
2?
3?
1?
7?
3?
7?
6?
6
5?
7
2
2
47
3?
7?
7?
5?
2?
1?
1?
4
1
7?
3?
7
48
7?
4?
5?
6?
3?
3?
6?
6
5
4?
2
1
49
2?
2?
5?
1?
5?
7?
7?
5
5
3
6
2?
50
5?
6?
1?
7?
2?
7?
4?
3
6?
5
2
3
51
2?
4?
7?
3?
2?
6?
7?
6
4
1
7?
7
52
5?
3?
5?
7?
5?
4?
6?
2?
1
7
7
4?
53
4?
2?
6?
6?
1?
5?
7?
4
3
1
1
4?
54
1?
1?
6?
3?
1?
6?
2?
3?
7?
3
1?
5
55
4?
2?
6?
4?
7?
1?
6?
6
2
4?
1
6?
56
1?
3?
4?
1?
3?
2?
3?
7?
7
1?
4?
7
57
6?
4?
7?
7?
6?
5?
4?
7
3
1?
1
1
58
4?
5?
6?
7?
6?
7?
3?
1
3
6
7?
2
59
5?
2?
4?
7?
5?
3?
6?
4
7
7
4
1?
60
4?
5?
2?
6?
5?
1?
5?
2?
3?
7?
7
5
61
1?
6?
3?
5?
4?
7?
3?
2
3
7?
1
7
62
1?
7?
5?
2?
5?
6?
5?
5
2
4?
4
4
63
7?
6?
1?
4?
3?
4?
5?
7
1
2
5
2?
64
3?
2?
7?
6?
1?
5?
7?
4?
4
6
4?
4
65
6?
6?
7?
5?
4?
2?
2?
2?
5
7
1
3
66
3?
6?
3?
3?
2?
4?
7?
3
3?
4
5
2?
67
4?
2?
7?
5?
4?
6?
2?
3
1
4
5
1
68
4?
3?
5?
5?
3?
3?
3?
1
3?
1
7?
5?
69
2?
7?
3?
1?
4?
5?
2?
3
6?
4?
3
3
70
2?
3?
1?
3?
3?
3?
3?
5
2
4?
7
3
71
4?
7?
5?
5?
5?
6?
7?
1
7?
4?
1
6
Scoring
You get 7,000 points for each level and 70,000 points every time you clear the grid. You also get points when discs explode but the number of points depends on the "chain" length of your explosions. The formula for determining the value of a chain explosion is 7*n^2.5 where 'n' is the chain length.
Here is a table that shows the value of a disc for a given chain length along with the total score for that chain...assuming only one disc exploded at each level of the chain. The longest chain I have been able to find is x29.
Chain
Length
Disc Score
Total Score
1
7
7
2
39
46
3
109
155
4
224
379
5
391
770
6
617
1387
7
907
2294
8
1267
3561
9
1701
5262
10
2213
7475
11
2809
10284
12
3491
13775
13
4265
18040
14
5133
23173
15
6099
29272
16
7168
36440
17
8341
44781
18
9622
54403
19
11014
65417
20
12521
77938
21
14146
92084
22
15891
107975
23
17758
125733
24
19752
145485
25
21875
167360
26
24128
191488
27
26515
218003
28
29039
247042
29
31702
278744
30
34506
313250
31
37454
350704
32
40548
391252
33
43790
435042
34
47184
482226
35
50730
532956
36
54432
587388
37
58291
645679
38
62309
707988
39
66490
774478
40
70835
845313
41
75345
920658
42
80024
1000682
43
84872
1085554
44
89893
1175447
45
95088
1270535
46
100459
1370994
47
106008
1477002
48
111738
1588740
49
117649
1776389
The Robot
I should be clear here that the robot I built doesn't make any decisions on its own about where to play the disc, the robot can't see the screen and therefore can't play the game by itself. My Drop7 simulator figured out what sequence to play the discs in to achieve a high score, I just programmed the robot to touch the screen in a specific pattern so that the discs would be played in that sequence.
The Stylus
One tricky part of this project was figuring out how to get the robot to touch the screen in a way that would register with the iPad. iPad screens aren't pressure sensitive, they are capacitive screens so you can't just stick a stylus in a robot's hand and have it work. I tried grounding a stylus but that didn't work either. After a bit of googling I found where some people had used frozen sausages as a stylus and that made me wonder if that would work for a robot since it is made of flesh like your finger. I was all out of those little breakfast sausages but I did have some hot dogs in the fridge so I gave one of those a try and BINGO!
Now there are a few downsides here
Hot dogs touch a large part of the screen at once compared to a normal stylus
Hot dogs dry out and the touch doesn't always register
Your iPad smells like a hot dog at the end of the day
If you think little kid fingers can make your iPad screen disgusting, try touching it with a hot dog a few hundred times.
I eventually figured out that when I had tried grounding a regular stylus I wasn't grounding it properly and that is why it didn't work. You have to take the little rubber tip off the end of the stylus, wrap some wire around the metal there, then put the rubber tip back on. Ground the wire and that should do the trick.
Hardware
I went through a few different styles of robots before settling on the one used in the video. My original robot is the one you see in the hot dog photo. It had problems consistently dropping discs in column 1 and column 7 though because the angle was so narrow by the time the motor moved the stylus that far to one side.
Next I tried building a little rover that would drive forward and backwards to the appropriate column. This wasn't accurate enough though because if the wheels slipped at all it would end up dropping the disc on the wrong column.
I decided I needed a platform that moved on a track so I could get the stylus exactly over the middle of a Drop7 column. I'll post some instructions later on how to build the robot. The only parts I used that did not come with the Lego Mindstorms kit were the gear racks. These were used to move the platform from side to side.
Here you can see the gear racks underneath the platform. A motor turns a gear which moves the gear rack which in turn slides the platform to the left or right on the track depending on which way I tell the motor to turn. Once the platform is centered over the right column, the motor on top of the platform turns a few degrees to touch the screen with the stylus then reverses direction to pick the stylus up.
Software
I am completely new to the world of Lego Mindstorms but I have to say it is amazing they are as popular as they are given how horrific the NXT-G software is that Lego provides. It is slow as Christmas, it locks up, it crashes, things that once worked randomly stop working, etc. Needless to say I didn't use it for very long until I started googling for an alternative. I settled on BricxCC which allows you to program Lego Mindstorm in NXC "Not eXactly C". If you have any programming experience at all I would say go with BricxCC over NXT-G, it will save you a lot of frustration.
My BricxCC code for this project is also on github.
The End
This was a lot of fun. I hadn't written a line of C in 6 years and had never used Lego Mindstorms so I learned a lot via this little project :)
