LLM-Integrated Multivariable Calculus Course(calculus.academa.ai)
calculus.academa.ai
LLM-Integrated Multivariable Calculus Course
https://calculus.academa.ai/
9 comments
I agree. I really cringed when I viewed this platform. The potential in educational AI rests in its interactivity, quizzing and socratic dialogue. There are many great and engaging courses on Calculus taught by charismatic and interesting instructors (humans). And in this age of MOOC, all it takes is a handful of strong online accessible courses for there to be diminishing returns in trying to improve the quality of lecture content.
I want to see agents quiz me, understand my strengths and weaknesses, setup study plans, using spaced repetition to ensure I retain information, engage in dialogue such that I come to the answer through articulated reasoning. Not watch lecture content similar to what's already out there but with less soul.
I want to see agents quiz me, understand my strengths and weaknesses, setup study plans, using spaced repetition to ensure I retain information, engage in dialogue such that I come to the answer through articulated reasoning. Not watch lecture content similar to what's already out there but with less soul.
I couldn't agree more. But if you look at it from this perspective, you may understand why we use static videos for this demo.
Definition: the AI tutor is some AI that can generate these videos.
If that generation happens in real time, it's a complete reimagination of education. It's literally the AI private tutor.
But before you can do that in real time, ask yourself: let alone real time, can you even generate a video, let's say, in 30 minutes? That's what we explored, and we found some great things. But it's not real time yet. The only practical way to publish what we'd built, though, was as a static video paired with chat.
The complete reimagination of education is on the way.
Definition: the AI tutor is some AI that can generate these videos.
If that generation happens in real time, it's a complete reimagination of education. It's literally the AI private tutor.
But before you can do that in real time, ask yourself: let alone real time, can you even generate a video, let's say, in 30 minutes? That's what we explored, and we found some great things. But it's not real time yet. The only practical way to publish what we'd built, though, was as a static video paired with chat.
The complete reimagination of education is on the way.
The impact of LLMs on learning is also something I am keen on, both for myself and for my children and how I should guide them along. The tech is undeniably useful but useless/harmless if employed without care for first principles of learning.
Most literature seem to indicate that whatever the manner of use, "friction" while learning ultimately still helps. My first "experimental" touchpoint is using LLMs as a called "socratic tutor".
Edit: because fat fingered submit while typing this on a split keyboard
Most literature seem to indicate that whatever the manner of use, "friction" while learning ultimately still helps. My first "experimental" touchpoint is using LLMs as a called "socratic tutor".
Edit: because fat fingered submit while typing this on a split keyboard
This sort of thing makes me depressed. The videos are LLM slop. You're a confused student. You do a web search. You run into something like this. It looks high-quality and professional. It is dogshit which no one can understand. The result is a confused learner who feels bad about themselves.
At the same time, thousands of low-quality resources overwhelm good content.
Integrating with an existing resources like MOOCulus could add a ton of value, in contrast, but wouldn't have the promise of making the creators money.
Being able to create manim animations at scale is a value-add, but doesn't seem enough of a value-add to create a business which does anything other than active harm. But it seems to be trying to be one.
At the same time, thousands of low-quality resources overwhelm good content.
Integrating with an existing resources like MOOCulus could add a ton of value, in contrast, but wouldn't have the promise of making the creators money.
Being able to create manim animations at scale is a value-add, but doesn't seem enough of a value-add to create a business which does anything other than active harm. But it seems to be trying to be one.
Well you can goto any large library in the world and you will find a zillion books on Calculus. Part of the learning and developmental process is working out for yourself what is "good" and "bad". And that has a lot to do with what your background is and what problems you are using Calculus for. So I dont feel its a new issue. People who want something better learn to find it.
Most textbooks are bad too. It's worth looking at some sample content to find the good ones.
[deleted]
There are some incorrect premises here.
"The videos are LLM slop." - No, they're not. Watch them.
"It's dogshit that no one can understand." - That's not true. Try it.
"The videos are LLM slop." - No, they're not. Watch them.
"It's dogshit that no one can understand." - That's not true. Try it.
I watched some snippets, and would like to put forward some points in support of the former:
- The voice is clearly TTS, which I think really loses something. Not having variation or stressing particular parts with intonation is a big deal in teaching. The beginning of lecture 5 has a pause in a weird spot (surface).
- The intro to the first lecture is already unnatural: "And that's of course now we're going to start working with functions of more than one variable."
- The scaffolding in lecture 1 is off to a bad start. The lecture tries to rope the student in with a question, but the question uses the notion and notation of a vector that has not been introduced yet. This reeks of a prompt "use a (motivating) question to introduce the topic", but a question that cannot be understood by a student does not help.
- The voice is clearly TTS, which I think really loses something. Not having variation or stressing particular parts with intonation is a big deal in teaching. The beginning of lecture 5 has a pause in a weird spot (surface).
- The intro to the first lecture is already unnatural: "And that's of course now we're going to start working with functions of more than one variable."
- The scaffolding in lecture 1 is off to a bad start. The lecture tries to rope the student in with a question, but the question uses the notion and notation of a vector that has not been introduced yet. This reeks of a prompt "use a (motivating) question to introduce the topic", but a question that cannot be understood by a student does not help.
These are 14 hours of video, so it's easy to find mistakes. We've probably made plenty of them. But this isn't AI slop.
We created the technology so that the content can be updated and improved over time. If we make mistakes, we'll fix them and continue maintaining the material. These aren't static MP4 files recorded once with a camcorder.
We created the technology so that the content can be updated and improved over time. If we make mistakes, we'll fix them and continue maintaining the material. These aren't static MP4 files recorded once with a camcorder.
To give a more precise indication, I watched maybe the first 5 minutes of lecture 1 and 5 for this.
I highly encourage you to take "didicatically useful" as a bar to aim for, rather than "not making mistakes".
I think it's a cool tech demo, but if you're positioning this as a course that's ready to be consumed by students, I think doing the quality control is not optional.
I highly encourage you to take "didicatically useful" as a bar to aim for, rather than "not making mistakes".
I think it's a cool tech demo, but if you're positioning this as a course that's ready to be consumed by students, I think doing the quality control is not optional.
Thank you for the feedback! We'll make it better. We take quality very seriously.
I did try it.
I'd provide feedback on how to make this better, but to be frank, I don't want to see this made better. I'd like open education made better. I'm on the other side here. Even if I did, I'm pretty sure you wouldn't listen.
In my career, I've done both open and closed models (more of the former in recently, more of the latter early career). I'm open to both, but education should be open, for a whole slew of reasons I can enumerate.
I'd provide feedback on how to make this better, but to be frank, I don't want to see this made better. I'd like open education made better. I'm on the other side here. Even if I did, I'm pretty sure you wouldn't listen.
In my career, I've done both open and closed models (more of the former in recently, more of the latter early career). I'm open to both, but education should be open, for a whole slew of reasons I can enumerate.
Content is interesting and seems accurate to me.
That narrator drives me nuts though. There too many “like” “you know” “so” “down here”. Comes off grating, I don’t think I could listen to multiple hours of this.
That narrator drives me nuts though. There too many “like” “you know” “so” “down here”. Comes off grating, I don’t think I could listen to multiple hours of this.
Thank you for the feedback!
The LLM has the video's full context, including visuals. Try chatting with it. Jump to any point in the video and ask questions about what you see.
Problem sets?
In terms of UI, it's just videos and an LLM chat. Though, the videos include problems along with their solutions.
I asked the bot questions about competitive Pokemon and Game of Thrones which it answered.
what even is competitive game of thrones?competing for who withstands more the finale?
Good, is there calc 1 and calc 2 or should we ask llm ?
Calc 1 and Calc 2 will come later. We'll actually be doing this for hundreds of technical subjects over the next 6–12 months.
How are you guaranteeing accuracy?
By storing the content in a semantically meaningful way (not mp4s), making it easy to review, version control, and maintain.
This isn't really an answer.
How are you checking the accuracy of the content? Especially if you plan to scale this to hundreds of courses, it doesn't seem like you actually have humans checking.
How are you checking the accuracy of the content? Especially if you plan to scale this to hundreds of courses, it doesn't seem like you actually have humans checking.
Let me elaborate.
Our content is text, not MP4 files. So we review it with humans and AI, just like how software companies review their software. We may make mistakes, but that's okay because we maintain the content. We continuously improve it and fix issues as we find them.
It'll even become more accurate than cam-recorded lecture videos over time, because those aren't being updated.
Our content is text, not MP4 files. So we review it with humans and AI, just like how software companies review their software. We may make mistakes, but that's okay because we maintain the content. We continuously improve it and fix issues as we find them.
It'll even become more accurate than cam-recorded lecture videos over time, because those aren't being updated.
What tool was used for video generation?
We have an in-house computer graphics pipeline. We reimplemented 3Blue1Brown's Manim in Rust from scratch. It is available at https://studio.academa.ai. The renderer runs in the browser using WebGPU.
is that renderer open source?
No, not yet. It's something we may do in the future.
LLM-written clones of free software into proprietary seem to be a growing trend now.
With manim, it's probably legal (I think MIT license?), but in general, it makes the world a worse place.
With manim, it's probably legal (I think MIT license?), but in general, it makes the world a worse place.
It's the AI frontier ethos. "I get mine." That's the culture now.
Speaking of which, it happened before, too, in the 1970's with personal computer software.
Speaking of which, it happened before, too, in the 1970's with personal computer software.
All we want to do is innovate. We simply can't open source this because we're rewriting and rebuilding it every day.
> All we want to do is innovate.
That's very clear. That's why we have spyware, ad tech, WWII-era German chemical weapons, the East India Company, Oracle's business model, fintech, ...
Ask anyone 40 years older. Regrets focus on time with family and meaningful work. Few people think: "Did I innovate enough?"
> We simply can't open source this because we're rewriting and rebuilding it every day.
I don't quite see the logical connection in this sentence.
Everything I've worked on in the past 15 years is open-source. I suspect you'd:
- Be able to find modest amounts of funding to do this in the open space (e.g. within the academy, as a teaching professor, adjunct, or similar) -- enough to sustain yourself, but without a promise of riches
- Be able to "innovate," make more E($), and do less harm in corporate America
That's very clear. That's why we have spyware, ad tech, WWII-era German chemical weapons, the East India Company, Oracle's business model, fintech, ...
Ask anyone 40 years older. Regrets focus on time with family and meaningful work. Few people think: "Did I innovate enough?"
> We simply can't open source this because we're rewriting and rebuilding it every day.
I don't quite see the logical connection in this sentence.
Everything I've worked on in the past 15 years is open-source. I suspect you'd:
- Be able to find modest amounts of funding to do this in the open space (e.g. within the academy, as a teaching professor, adjunct, or similar) -- enough to sustain yourself, but without a promise of riches
- Be able to "innovate," make more E($), and do less harm in corporate America
First, I need to make my startup successful so I can continue innovating over the next 15 years. Open-sourcing our work could be a distraction at this stage. Maintaining an open-source project requires significant time and effort.
It would also make it easier for competitors to build products that compete directly with us, such as https://studio.academa.ai. That would make it harder for my startup to succeed.
I'm not building this for the sake of making money. I simply need enough protected space to innovate competitively and give the company the best chance to succeed. If it does, I'll be able to keep building and pushing education forward for years to come.
It would also make it easier for competitors to build products that compete directly with us, such as https://studio.academa.ai. That would make it harder for my startup to succeed.
I'm not building this for the sake of making money. I simply need enough protected space to innovate competitively and give the company the best chance to succeed. If it does, I'll be able to keep building and pushing education forward for years to come.
Don't pay too much attention to the naysayers if they don't have constructive advice to offer (but do try to collect what signal you can from the noise). There are a lot of people who viscerally hate AI for a ton of different reasons, or maybe are perfectionists about the subject matter. Even if these AI technologies are currently imperfect, they solve Bloom's Two Sigma problem - students tutored to mastery one-on-one perform two standard deviations better than students taught in traditional classroom settings. Imagine a world where everyone is 30 IQ points smarter just from better education technology!
Personally, I just used chatgpt the other day to _finally_ understand why a dot product of vectors is equivalent to cosine similarity. Mathacademy's lesson is very good but the explanation wasn't enlightening for me, it took chatgpt breaking it down into tiny bite-sized chunks, patiently re-explaining what a cosine actually is and where it disappears to in the dot product, making little animations of vectors with sliders that I could move myself to try stuff with differently shaped and rotated triangles, for me to _actually and viscerally understand it_ and after spending probably 2 hours on this simple concept, now I _actually_ feel like I understand it rather than having just memorized it. So I feel much more confident that I can build deeper understanding on top of that.
Personally, I just used chatgpt the other day to _finally_ understand why a dot product of vectors is equivalent to cosine similarity. Mathacademy's lesson is very good but the explanation wasn't enlightening for me, it took chatgpt breaking it down into tiny bite-sized chunks, patiently re-explaining what a cosine actually is and where it disappears to in the dot product, making little animations of vectors with sliders that I could move myself to try stuff with differently shaped and rotated triangles, for me to _actually and viscerally understand it_ and after spending probably 2 hours on this simple concept, now I _actually_ feel like I understand it rather than having just memorized it. So I feel much more confident that I can build deeper understanding on top of that.
Thank you!
Is cloning Manim "innovation" now?
Plenty of open source is updated every day.
You had no problem taking advantage of Manim to profit, but when it comes to returning the great favor done to you by Manim being open source, you don't care?
Okay.
Plenty of open source is updated every day.
You had no problem taking advantage of Manim to profit, but when it comes to returning the great favor done to you by Manim being open source, you don't care?
Okay.
Everything I do is driven by a single goal: to make my startup successful so I can innovate in edtech. Nothing else.
You may not believe me, and you may think I have ulterior motives, but I don't. My only goal is to build technology that meaningfully advances education.
You may not believe me, and you may think I have ulterior motives, but I don't. My only goal is to build technology that meaningfully advances education.
> Everything I do is driven by a single goal: to make my startup successful so I can innovate in edtech. Nothing else.
I believe you entirely. That comes across in everything you say. Huge harm has been done in education precisely by people with similarly amoral motivation. History of education is filled with case after case of amoral motivations leading to immoral behavior.
Other industries have this more extreme. Fritz Haber is a biography worth reading.
The comments you're making aren't very honest -- I suspect with yourself more than with anyone else. For example:
> Maintaining an open-source project requires significant time and effort.
No. It doesn't. You need to flip the "public" bit on github and post a "caveat emptor" shingle. The effort you're cloning -- manim -- did exactly that. The result was a synergistic, friendly, manim-ce.
And the author has been able to work full-time in this space, his entire career, without doing anything evil, unethical, harmful, or unpleasant, and strictly advancing the world.
You're competing -- rather than cooperating -- with folks like him.
> I simply need enough protected space to innovate competitively and give the company the best chance to succeed.
You don't seem to understand how this works. A few points:
- Education is a highly-regulated industry for good reasons (student harm, societal harm, privacy risks, etc.), and education research goes through IRBs for similar reasons.
- Startups can work on IP, but usually -- and especially in education -- networks, connections, support, etc. are more important. It's hard to build an IP moat.
- Startups have a 95% failure rate. I'd give this one more than that.
A few corollaries:
a) With the current model, you'll have a hard time building partnerships with the people you need, and indeed, people (myself included!) will be rooting for you to fail. That weakens, not strengthens, your odds of success.
b) An example of another way to accomplish your stated goal is a five-year research grant, or a position which supports doing this kind of work (e.g. instructional staff at your university). Those have >> 5% success rates.
c) Open business models aren't less likely to be successful here than closed ones. Red Hat sold for $34B.
Otherwise, my best wish for you is to fail fast, and to learn a lot from the process.
I believe you entirely. That comes across in everything you say. Huge harm has been done in education precisely by people with similarly amoral motivation. History of education is filled with case after case of amoral motivations leading to immoral behavior.
Other industries have this more extreme. Fritz Haber is a biography worth reading.
The comments you're making aren't very honest -- I suspect with yourself more than with anyone else. For example:
> Maintaining an open-source project requires significant time and effort.
No. It doesn't. You need to flip the "public" bit on github and post a "caveat emptor" shingle. The effort you're cloning -- manim -- did exactly that. The result was a synergistic, friendly, manim-ce.
And the author has been able to work full-time in this space, his entire career, without doing anything evil, unethical, harmful, or unpleasant, and strictly advancing the world.
You're competing -- rather than cooperating -- with folks like him.
> I simply need enough protected space to innovate competitively and give the company the best chance to succeed.
You don't seem to understand how this works. A few points:
- Education is a highly-regulated industry for good reasons (student harm, societal harm, privacy risks, etc.), and education research goes through IRBs for similar reasons.
- Startups can work on IP, but usually -- and especially in education -- networks, connections, support, etc. are more important. It's hard to build an IP moat.
- Startups have a 95% failure rate. I'd give this one more than that.
A few corollaries:
a) With the current model, you'll have a hard time building partnerships with the people you need, and indeed, people (myself included!) will be rooting for you to fail. That weakens, not strengthens, your odds of success.
b) An example of another way to accomplish your stated goal is a five-year research grant, or a position which supports doing this kind of work (e.g. instructional staff at your university). Those have >> 5% success rates.
c) Open business models aren't less likely to be successful here than closed ones. Red Hat sold for $34B.
Otherwise, my best wish for you is to fail fast, and to learn a lot from the process.
Well, OpenAI was "not for profit" and still has "open" in the name. It's the age of transparent doublespeak.
Why can’t you do that in the open? Normally I don’t think people have a moral obligation to open source, but if you’re using AI to reimplement an existing open source library I think you do.
Who is signing off on the correctness?
We are. We're confident in the quality of the teaching and the accuracy of the content in these videos.
Our videos are not some MP4 files sitting on a disk. We see our content like software. We review it, version-control it, and maintain it.
Our videos are not some MP4 files sitting on a disk. We see our content like software. We review it, version-control it, and maintain it.
Who is "we"?
We're an edtech startup, Academa: https://academa.ai
We're two PhD students and close friends since our undergraduate years.
I'm Sina: https://sinaatalay.com
And my co-founder, Apo: https://geduk.io
We're two PhD students and close friends since our undergraduate years.
I'm Sina: https://sinaatalay.com
And my co-founder, Apo: https://geduk.io
[deleted]
As someone who has self-taught most of my skills both before and after AI, some deep feedback: I don't want a fixed piece of content when learning with AI, like a video or blogpost or book—unless I'm completely new to the subject, and even then maybe not.
The reason is that some parts of the topic will be naturally easier or harder for me. When I use AI I tell it everything I know and understand and start working from my most burning questions and misunderstandings. This lets me cover the maximum amount of non-redundant ground in regards to my understanding.
We have this amazing new technology and you're conforming it to models of schooling (like the Prussian model) which are one or more centuries old. The technology is so powerful that it should allow you to completely reshape education, not merely replicate the status quo.