Couple days ago, i used Claude to implement an improved version of gpt-1. I am no ML Engineer by no means. I am just a normal backend engineer. I ended up creating a hybrid between gpt-1 and modded-nanogpt (from KellerJordan).
I was able to reproduce the results of the original gpt-1 paper with my gaming PC. I don't even have alot of VRAM. My NVIDIA GeForce RTX 2060 SUPER was able to reproduce most of the results with just 1 hour of training. I would totally recommend to do the same, if you are interested in pre-training LLMs.
Thank you for your kind words. Yes, i hate subscriptions. But, also. Renovating is a one-off thing and i can't justify charging a monthly fee for a service that is only needed once.
This is one of the reasons people love using xona: no subscriptions :)
i was thinking the same. And i am sure it is in the training set, but i couldn't solve 10 hard problems. Also, most of the hard problems, the LLMs weren't able to solve in one shot. How do you explain this?
I'm working on Xona AI (https://xona.ai). I want to democratize the generation of 3D visualizations for interior and exterior design. We will launch next week on ProductHunt :)
Isn't it depressing, that we live in 2023 and the predominant document format is pdf, which was invented in 1993 and is optimized for printing? I would love to have a new format, which is easily parseable (like JSON) AND printable (like PDF).
This is a perfect example of how AI is taking over the world by storm. I don't know how people don't realize that there will be no jobs left for billions of people.
Yes, billions. Not Millions.
I don't quite get the "New Jobs will be created" fallacy.
Let me explain:
What is job? An Abstract way of looking at it: A job is something that requires a set skills to accomplish a task.
What most politicians don't get: Researchers like OpenAi teach machines SKILLS not jobs.
A little thought experiment:
Let's say humans are capable of 100 skills.
Skills can be anything from: driving, seeing, hearing, reading, walking, carrying, drawing etc.
Usually, a low paying job requires little to no traning.
For example: Someone in a warehouse that picks the stuff you have ordered. The skill that are required are: walking, picking and using a device.
A High paying jobs usually requires more skills and/or experience.
We train machines to see better, hear better, sort faster etc.
Any new job will require some sort of skills out the set of skills that can be trained.
But the moment you create this job, it will be automated, because a machine can do it better and faster.
We need to adress this now, otherwise i don't see a bright future for the generations to come.
I agree. It is very unsettling to hear that a stranger "stalked the st out of you" and i would prefer direct help.
> Regarding suicidal thoughts, they ARE normal for many people and perfectly rational given their conditions and length of suffering. It's not an attitude problem or something therapy or pills can fix. Sometimes those things can help COPE temporarily but it's not a fix of the root cause. I know it's hard to understand that if you have never been there and there was a time I would swear I could NEVER think that was valid but experiences changes you. In my view nobody wants to die, even when they claim so, they just want the pain to end and can't find a way here in life...sometimes because there simply IS no way...others because they can't get the real and tangible help they need.
i agree alot. I had a hard time in my 20's. My parents live in another country. I was living in this "new" city on my own with problems i couldn't handle. I was always on the brink of being homeless.
I didn't want to take counseling mainly because they wouldn't fix the root of my problems: No income, but a lot of debts.
What if i took 3 Months of therapy? I would still have my debts. It is important to find the root cause and tackle it directly.
Don't get me wrong: Therapy can help alot. But the therapist is not going to get a job for you.
I had to do it on my own. It was even hard to get out of bed for me. But i did. Because of the wake up call.
We live in a society where 90% of the people have to work for their status. If you don't work, you don't have money. I don't like the idea of "making a living". Living should be a human right. But the reality is: it isn't. You have to work in order to have a place to sleep or get food. Living on Food stamps only keeps you alive. You are not part of society. In hindsight, i did miserable comparing to all my friends (most of them PhD students). But i am a huge success comparing to the vast majority of people who are living in 3rd world countries. Back then i had forgotten how lucky i am to be born in a rich country. Today i still live a simple life, cause i use every extra penny to pay my debts, but i am grateful for what i have! And one day, i can have a vacation. Something i didn't have in 8years.
you are right, the police was not very professional and it might have ended badly. Luckily it didn't. First i was scared that my ex send them to me and i was in trouble (althoug i did nothing illegal). Then i was relieved that it was about my post on HN.
Reading this made me uncomfortable, but also happy. A Stranger from the internet cared (about my life) and took action. I got 80 messages on that thread. Most of them were upliftig. But taking action is another level. Thank you for that! Faith in humanity restored.
You could have reached out to me directly. I would be happy to have a coffee with a fellow HN and "talk about it". I'd rather talk to a stranger with an open ear than a psychologist who listens because he/she gets paid for it.
If there is anything i can do for you, you can contact me at: [email protected] .
I was able to reproduce the results of the original gpt-1 paper with my gaming PC. I don't even have alot of VRAM. My NVIDIA GeForce RTX 2060 SUPER was able to reproduce most of the results with just 1 hour of training. I would totally recommend to do the same, if you are interested in pre-training LLMs.
The code is here: https://github.com/epoyraz/modded-gpt-1 But, you can also just ask Claude 4.8 or Codex 5.5