Sorting 2 Tons of Lego (2017)(jacquesmattheij.com)
jacquesmattheij.com
Sorting 2 Tons of Lego (2017)
http://jacquesmattheij.com/sorting-lego-many-questions-and-this-is-what-the-result-looks-like/
11 comments
Crazy how often this has been posted to HN.
I think it's one of the 2 examples where Machine Learning actually was seen usefull as DIY-project.
The other one is with a cucumber farm: https://cloud.google.com/blog/products/gcp/how-a-japanese-cu...
Fyi: The real reason why i posted it, is that suddenly your RSS feeds on your website work again i suppose :P
So all your items on your blog are suddenly in my list again :)
That and ... it's an awesome story :p
The other one is with a cucumber farm: https://cloud.google.com/blog/products/gcp/how-a-japanese-cu...
Fyi: The real reason why i posted it, is that suddenly your RSS feeds on your website work again i suppose :P
So all your items on your blog are suddenly in my list again :)
That and ... it's an awesome story :p
> The real reason why i posted it, is that suddenly your RSS feeds on your website work again i suppose :P
I fixed a minor bug in my Hugo setup and posted a new article, that's probably the reason why that changed. Now I still need to go fix the images on older posts. No rest for the wicked!
My current project also has ML in it; music, MIDI, programming, what's not to like :)?
I'm very much unsure about whether or not I will be able to get it to work though, I may have bitten off more than I can chew with this one.
I fixed a minor bug in my Hugo setup and posted a new article, that's probably the reason why that changed. Now I still need to go fix the images on older posts. No rest for the wicked!
My current project also has ML in it; music, MIDI, programming, what's not to like :)?
I'm very much unsure about whether or not I will be able to get it to work though, I may have bitten off more than I can chew with this one.
Ahhh, yes. It could be that the URL or Unique Id changed of the feed.
No worries for the images though, as i'm not using that ( but i could)
http://handlr.sapico.me/ :p
No worries for the images though, as i'm not using that ( but i could)
http://handlr.sapico.me/ :p
Can confirm, Feedly handed me a bunch of older posts as unread along with the "Briefcase" on earlier today...
What's the current project? I've been wanting to post one of mine for the past few years, but I never get around to it...
Trying to create a very patient (and cheap) piano teacher. I realize it will never be a substitute for a real teacher but if it gets me 50% there that's already quite neat. But this is much harder than I thought it would be.
Lol, i was thinking about the usefullness of Music, MIDI and AI. That i have no relation to it.
BUT, this is probably the only thing that i want to do with Music - learning how to play the piano.
Nice one!
BUT, this is probably the only thing that i want to do with Music - learning how to play the piano.
Nice one!
What's hard about it? Can you share some more details? Are you using a led strip on the keys?
Polyphonic sound decomposition is pretty much an open problem. I have an 'interesting' idea on how to solve it. I'm sure I'll be accused of cheating if I get it to work ;)
Oh, that's true (Melodyne made a plug in that will do it, AFAIK). Can't you do something else, like optically or mechanically read the keys? Do you even need to read them? I guess you want to tell people if they made a mistake, hmm...
Something you could do would be IR LEDs and an IR camera, where the LEDs are only reflected on the shiny keys when they're pressed due to the angle.
Something you could do would be IR LEDs and an IR camera, where the LEDs are only reflected on the shiny keys when they're pressed due to the angle.
Hehe, 'why don't you' :)
Yes, there are many use cases that I have in mind where this is a must. In fact, if I can't crack it then it's a useless project (other than that I will learn from it).
There are all kinds of devices that could give you midi inputs (such as the 'Piano Bar', sadly discontinued), but that's the easy way out. After all, you could also simply use a MIDI enabled keyboard. But I want it to work with a real piano, as well as possibly other instruments.
The best software for this at the moment is Anthemscore.
Yes, there are many use cases that I have in mind where this is a must. In fact, if I can't crack it then it's a useless project (other than that I will learn from it).
There are all kinds of devices that could give you midi inputs (such as the 'Piano Bar', sadly discontinued), but that's the easy way out. After all, you could also simply use a MIDI enabled keyboard. But I want it to work with a real piano, as well as possibly other instruments.
The best software for this at the moment is Anthemscore.
Hmm, very true. Do you want it to not require modifications to the piano? I guess that would mean that you pretty much need to do sound decomposition...
Yes, zero hardware changes or mods to the piano. I think that would be a dealbreaker for many people. It also means that the barrier to entry would be much lower.
Hmm, yes, if you can get it to work with just sound, that would be trivial to set up. If not, I think the light idea would still be valuable, as technically you wouldn't be changing the piano (just adding a camera and light above it).
I liked "counting bees on a rasp pi with a conv net" too.
https://news.ycombinator.com/item?id=17103635
https://news.ycombinator.com/item?id=17103635
It's exactly the kind of things I hope to see every now on then on Hacker News! :)
I should set it up again and give a live demo one of these days. It's lots of fun to watch it work. At least once every week someone mails me about it. The bulk of these are wantrepreneurs who have suddenly seen the light and want to take the project over, and who think I'm gullible enough to believe their cover stories (5 minutes of Googling usually turns up a commercial operation). Can't blame them for trying to corner the market I guess.
Meanwhile, the consultancy I've been running for the last 12 years is growing up, rapidly. We just named our first CEO (other than me), who will hopefully take it to the next level so who knows, I just might have some free time again in the near future. Here's to hoping.
Meanwhile, the consultancy I've been running for the last 12 years is growing up, rapidly. We just named our first CEO (other than me), who will hopefully take it to the next level so who knows, I just might have some free time again in the near future. Here's to hoping.
Did you ever sell any of the sorted Lego?
Yes, quite a bit actually, mostly to the United States and to one fellow here in NL that was building a huge castle.
https://twitter.com/jmattheij/status/1016382615417389057
Amazing accomplishment, he surveyed the building just prior to demolition with a drone and then built the whole thing without a single drawing.
https://twitter.com/jmattheij/status/1016382615417389057
Amazing accomplishment, he surveyed the building just prior to demolition with a drone and then built the whole thing without a single drawing.
For comparison:
Some photos of the castle: https://www.davidbakerphotography.com/location-history/resid... (see in particular one near the bottom)
About the demolition: https://www.byronhartshorn.com/2017/05/31/chateau-de-noisy-i...
Some photos of the castle: https://www.davidbakerphotography.com/location-history/resid... (see in particular one near the bottom)
About the demolition: https://www.byronhartshorn.com/2017/05/31/chateau-de-noisy-i...
That would have been one case where 'eminent domain' would have been a good solution.
It's a real pity to see such an iconic building go to waste. The demolition story tries to spin it hard as though the family had no other option, but of course they did: if they were unable to be good stewards of the property then they could have donated it to the Belgian state.
But after looting it for anything valuable it is of course much better to have a white-wash story out there about how those urban explorers violated the privacy of the owners (never mind the fact that it had been abandoned for years and that there was plenty of damage from neglect alone).
It's a real pity to see such an iconic building go to waste. The demolition story tries to spin it hard as though the family had no other option, but of course they did: if they were unable to be good stewards of the property then they could have donated it to the Belgian state.
But after looting it for anything valuable it is of course much better to have a white-wash story out there about how those urban explorers violated the privacy of the owners (never mind the fact that it had been abandoned for years and that there was plenty of damage from neglect alone).
I made a image search in Google and picked the two that looked more nice/relevant. It's a pity that the building was destroyed.
I bought some! (Thanks Jacques!!!)
Congrats :)
Thank you. Doing anything for a long time is hard, I can't wait to be free again to do other stuff. In 2016 when I did this project I still had lots of free time, we did 11 tech due dilligences that year. In 2017 there were 18, in 2018 43 and this year we will likely end up somewhere between 60 and 80 so it long ago surpassed what one person can do.
It is surprisingly hard to scale a one-man show.
It is surprisingly hard to scale a one-man show.
I agree with Nico its got a lot to like, it is very approachable and doesn't imply access to a lot of hard to get/use equipment. So good stuff.
Now you need sub-sorters :-) I always kept my technic stuff sorted by various parts (large gears, medium gears, small gears, motors, axles, wheels, etc)
Another interesting project would be a 'pick a part' machine that used an arm and image recognition to select parts from bins and assemble a 'kit' (all the parts needed to make a particular thing)
Now you need sub-sorters :-) I always kept my technic stuff sorted by various parts (large gears, medium gears, small gears, motors, axles, wheels, etc)
Another interesting project would be a 'pick a part' machine that used an arm and image recognition to select parts from bins and assemble a 'kit' (all the parts needed to make a particular thing)
> Now you need sub-sorters
The main limitations is to be able to sort into only 12 bins in one pass. If you want more detail or more groups then you need more passes. Ideally it would sort into a few hundred bins at once but I have not been able to come up with a geometry that works well. The most radical idea would be a tower of 2 meters high or so with bins spaced around it like flower petals, the air jet would have to be timed super precisely to aim the part into the right bin, especially near the bottom where the velocity would be quite high.
Pick a part is easier, the software is trivial but the hardware is anything but, an 'automated warehouse' for Lego parts would be a pretty tricky thing from a mechanical point of view.
And once you're on that train of thought, the obvious end-game is a machine that re-assembles the sets, then we can declare Lego a 'solved problem'.
The main limitations is to be able to sort into only 12 bins in one pass. If you want more detail or more groups then you need more passes. Ideally it would sort into a few hundred bins at once but I have not been able to come up with a geometry that works well. The most radical idea would be a tower of 2 meters high or so with bins spaced around it like flower petals, the air jet would have to be timed super precisely to aim the part into the right bin, especially near the bottom where the velocity would be quite high.
Pick a part is easier, the software is trivial but the hardware is anything but, an 'automated warehouse' for Lego parts would be a pretty tricky thing from a mechanical point of view.
And once you're on that train of thought, the obvious end-game is a machine that re-assembles the sets, then we can declare Lego a 'solved problem'.
Do you use relays for the blowers? Can you use MOSFETs for more accuracy?
Hm, never even thought that that might be a factor. Yes, it uses relays. I'm not sure how much belt travel will be off because of that, but given that it would be a constant it would likely already be accounted for a an offset to 'center of part' which is where I try to hit the parts to make sure they don't spin like mad.
That sounds reasonable, where you may have more luck with something more accurate is if you had to run the blower for a shorter time, or maybe pulse it to lower the wind speed (I'm not sure if that's possible).
The relay pulses are about 50 msec, that's mostly limited to ensure there is enough air to rapidly hit multiple bins in a short time, as well as to be able to 'pick out' a part from two adjacent parts on the belt without hitting the other two. That took a while to get right!
Have you ever posted the code? I would totally try implementing this, as the hardware work is something I can do, but my programming skills are rusty enough that I don't think I can write the code. I'd love to see other people try to build and improve on it.
No, I was planning to and then every 2 bit Lego trader and bricklink contacted me to try to go commercial with it. That's not how I envision this and so I decided to keep it inside for now. A ton of work went into labeling the dataset (many 10's of thousands of images), and there is no way I will let the likes of Bricklink and overpriced parts sellers use it to corner the market.
This is very much a mom-and-pop market, and there are countless people out there that augment their pensions by sorting Lego by hand and re-selling it.
Releasing this would put them out of business overnight, or, they'd have a grace period as long as someone does not put together industrial mechanics for it and then it is 'game over'.
This is very much a mom-and-pop market, and there are countless people out there that augment their pensions by sorting Lego by hand and re-selling it.
Releasing this would put them out of business overnight, or, they'd have a grace period as long as someone does not put together industrial mechanics for it and then it is 'game over'.
Fair enough. I guess I'll have to figure it out myself. :D
(2017)
The pictures at the end are so satisfying to see. I have a four year old and their Lego is a constant throwback to good times.
I think shipping to Russia will be very expensive for me. Also, the difference in exchange rates will play a big role. I apologize for the bad English. I am writing using Google translator
If you're ever in the neighborhood or know someone who travels to NL then let me know.
unfortunately I can only dream of visiting you)) for a month of work we are paid 355 euros. but if someone from my friends goes, I will inform you! it's so interesting to communicate with someone from another country)), albeit with the help of Google translator
Fyi, shipping to Russia:
- 1 kg = 30,03€
- 5 kg = 42€
- 30 kg ( max ) = 77,76€
- 1 kg = 30,03€
- 5 kg = 42€
- 30 kg ( max ) = 77,76€
Can you sort them with enough detail that you can reassemble existing sets? That’s probably the way to go for adding significant value back to the pieces
Yes, absolutely.
The workflow would be: sort about 10% of the pieces in a test, use that to determine which sets are present based on the rare pieces, then pull those out of the bulk in one pass, it can do 12 sets in one go. Then you do it all over again until there is no more left.
The workflow would be: sort about 10% of the pieces in a test, use that to determine which sets are present based on the rare pieces, then pull those out of the bulk in one pass, it can do 12 sets in one go. Then you do it all over again until there is no more left.
Given a large set of unsorted lego pieces, and a list of possible lego sets with associated dollars values:
Do a single pass to identify/scan all the pieces. Then based on that, divide those into sets such that the associated dollar values are maximized and such that the number of sets is minimized.
Then select the 11 most valuable of the sets and use the sorting machine to sort these out, all the remaining legos go in the 12 set. Repeat.
Simple ;D
Do a single pass to identify/scan all the pieces. Then based on that, divide those into sets such that the associated dollar values are maximized and such that the number of sets is minimized.
Then select the 11 most valuable of the sets and use the sorting machine to sort these out, all the remaining legos go in the 12 set. Repeat.
Simple ;D
This project is fantastic, both the goal and the execution really appeals to me. It screams hacker/maker, good job.
[deleted]
The sorting is colour first for me. Colour is the resource that matters, the limiting factor to a model. Therefore I find it amusing that this is not the primary consideration.
As a child I found that the models made by my sister and myself were in a different league to those of the other kids because of this aspect of colour. We could use every brick in the box to build a townscape but nothing in the scene would be built without attention to the colours. Some other house or building might have to be part demolished to get the right colour bricks, it came down to resource optimisation. Those other kids that didn't have the colours sorted out were beginner level, they had worked out the bricks stick together but hadn't worked out that colours matter, as they do with the Lego sets, which are never jumble sale coloured with random bricks everywhere. There always has to be this aspect of taste to the colours.
Right now I do have a manually sorted selection of bricks. They are for my niece when she is old enough to move on from Duplo. We have sorted the bricks by colour and then by size. The challenge to this has also involved washing them with the sorting being by colour and then size granularity. There are specials such as the flowers and other vegetation, doors/windows and so forth. In theory my niece will be able to get the bigger blocks first and then the specialised fiddly piece, e.g. 1x tiles, can come along later.
There have been other decisions made, for instance the mini-figures have been broken down to their component parts so that there is no knowing what they were originally. There are just lots of tops/bottoms/heads and hairdos.
We are short of roof parts. But who is to know if my niece would want to be making the town scene that was the de-facto model that my sister and myself would want to make?
I also think too many specialist brick shapes can be bad for the imagination, so I hope the staggered release of basic blocks to then move on to the smaller pieces will help with this aspect of creative possibilities for my niece. My niece is a single child so won't have resource wars with a sibling, which is a bit sad.
There are many potential customers, including those too old for LEGO (not that you can ever be too old for it), I hope my 'colour first' pointers help.
As a child I found that the models made by my sister and myself were in a different league to those of the other kids because of this aspect of colour. We could use every brick in the box to build a townscape but nothing in the scene would be built without attention to the colours. Some other house or building might have to be part demolished to get the right colour bricks, it came down to resource optimisation. Those other kids that didn't have the colours sorted out were beginner level, they had worked out the bricks stick together but hadn't worked out that colours matter, as they do with the Lego sets, which are never jumble sale coloured with random bricks everywhere. There always has to be this aspect of taste to the colours.
Right now I do have a manually sorted selection of bricks. They are for my niece when she is old enough to move on from Duplo. We have sorted the bricks by colour and then by size. The challenge to this has also involved washing them with the sorting being by colour and then size granularity. There are specials such as the flowers and other vegetation, doors/windows and so forth. In theory my niece will be able to get the bigger blocks first and then the specialised fiddly piece, e.g. 1x tiles, can come along later.
There have been other decisions made, for instance the mini-figures have been broken down to their component parts so that there is no knowing what they were originally. There are just lots of tops/bottoms/heads and hairdos.
We are short of roof parts. But who is to know if my niece would want to be making the town scene that was the de-facto model that my sister and myself would want to make?
I also think too many specialist brick shapes can be bad for the imagination, so I hope the staggered release of basic blocks to then move on to the smaller pieces will help with this aspect of creative possibilities for my niece. My niece is a single child so won't have resource wars with a sibling, which is a bit sad.
There are many potential customers, including those too old for LEGO (not that you can ever be too old for it), I hope my 'colour first' pointers help.
Sorting color is problematic. Yes, it is an easy first sort step with relatively limited number of output bins. But once sorted by color it becomes 10 times at least harder to pick out a shape. Anybody that I know that has sorted Lego in quantity first goes for that because it seems so obvious, and then after a while all of them revert.
Think about it: pick the 2x4 brick in red from a pile of 2x4 bricks in various colors vs pick the 2x4 brick in red from a pile of red bricks in various shapes. The first you can do in a heartbeat, the second will take much longer, and it is also much harder to determine that a piece isn't there in the first place.
Think about it: pick the 2x4 brick in red from a pile of 2x4 bricks in various colors vs pick the 2x4 brick in red from a pile of red bricks in various shapes. The first you can do in a heartbeat, the second will take much longer, and it is also much harder to determine that a piece isn't there in the first place.
How do you disassemble the Lego when they are first acquired? My guess is that, buying bulk Lego, some of them will already be snapped together. Is that a manual first step, or do you have a technique to separate them automatically?
That's an unsolved problem. You'd be surprised how hard it is to separate certain parts from each other. But the first bulk sorting step usually takes care of picking out the larger assemblies and there is a class 'assembly' which gets picked out all by itself by the machine. That at least allows you to concentrate on the parts that are still stuck together.
Can the machine detect three identical plates stuck together? Just curious.
My most hated parts to separate are a flat dot on top of a regular dot. Second runner-up is the 1x1 plate on top of a dot, which is common in the Minecraft sets. I'd pay for a machine just to separate these for me!
My most hated parts to separate are a flat dot on top of a regular dot. Second runner-up is the 1x1 plate on top of a dot, which is common in the Minecraft sets. I'd pay for a machine just to separate these for me!
Depending on the viewing angle: yes. If the part should land particularly unfavorably on the belt then there is a small chance that it would not be seen.
1-2 years ago, I commented on HN in a random Lego thread and Jacques sent my son (we're in Australia) some bulk Technic pieces. As with all Lego, they've been played with over and over and I always remember that generosity.
Thanks Jacques.
Thanks Jacques.
I bought 2kg from Jacques, also delivered to Australia, back in (searches email) Aug 2017.
It has brought much joy to various small people (and me...)
It has brought much joy to various small people (and me...)
The machine is impressive. It’s so nice to see hardware project here.
2 tons LEGO has eBay Kleinanzeigen value of €20k+ if one sells them in packets of 5 kg. My problem in LEGO building is simple: I don’t have enough rare parts. These can be substituted with improvised modules, but when set needs 3-5 of them it’s not buildable anymore. Having many improvised modules break original plan and the result sometimes isn’t satisfactory anymore.
2 tons LEGO has eBay Kleinanzeigen value of €20k+ if one sells them in packets of 5 kg. My problem in LEGO building is simple: I don’t have enough rare parts. These can be substituted with improvised modules, but when set needs 3-5 of them it’s not buildable anymore. Having many improvised modules break original plan and the result sometimes isn’t satisfactory anymore.