vikp · HackerTrans

vikp

no profile record

Submissions

Show HN: Surya – OCR and line detection in 93 languages

github.com

11 pointsby vikp2 tahun yang lalu1 comments

Show HN: Texify – OCR math images to LaTeX and Markdown

github.com

21 pointsby vikp3 tahun yang lalu3 comments

A vision for the AI web: the real web 3.0

vikas.sh

15 pointsby vikp3 tahun yang lalu10 comments

Build a semantic search engine in Python

vikas.sh

1 pointsby vikp3 tahun yang lalu1 comments

Show HN: Endless – Learn anything with personalized AI and blocks

endless.academy

1 pointsby vikp3 tahun yang lalu0 comments

Show HN: Researcher – answer questions using Google and GPT-3

github.com

1 pointsby vikp4 tahun yang lalu1 comments

Introduction to machine learning using k-NN and Airbnb data

dataquest.io

3 pointsby vikp9 tahun yang lalu0 comments

An interactive introduction to Spark, using Python

dataquest.io

4 pointsby vikp11 tahun yang lalu0 comments

Predicting HN upvotes from headlines with python

blog.dataquest.io

10 pointsby vikp11 tahun yang lalu2 comments

A collaborative machine learning contest

github.com

2 pointsby vikp11 tahun yang lalu0 comments

Using naive Bayes to predict movie review sentiment

blog.dataquest.io

51 pointsby vikp11 tahun yang lalu8 comments

I barely graduated college, and that's okay

vikparuchuri.com

95 pointsby vikp11 tahun yang lalu60 comments

Clustering US Senators with k-means

blog.dataquest.io

87 pointsby vikp11 tahun yang lalu10 comments

Learning to code kept me sane when I was a diplomat

vikparuchuri.com

100 pointsby vikp12 tahun yang lalu46 comments

Hipvote, gamifying congress since yesterday

hipvote.us

3 pointsby vikp12 tahun yang lalu1 comments

Show HN: Turf.ly, our weekend hackathon project. Sign up to help

turf.ly

2 pointsby vikp12 tahun yang lalu1 comments

Show HN: Scan, a simple automated essay scoring server

github.com

1 pointsby vikp12 tahun yang lalu0 comments

Show HN: ShareMyTime, an easy way for others to book your time

sharemyti.me

1 pointsby vikp12 tahun yang lalu0 comments

Making music automatically with markov chains and genetic algorithms

vikparuchuri.com

4 pointsby vikp13 tahun yang lalu2 comments

How much do characters on the Simpsons like each other?

vikparuchuri.com

2 pointsby vikp13 tahun yang lalu0 comments

comments

23 hari yang laludiscuss

It's an interesting illustration of the state of the AI market that immediately after arguing that AI cannot do anything complex...we have an ad arguing that AI can actually do those things. Even the people telling you that AI is a hype cycle buy into the hype cycle.

First, we have this section:

But the noise continues on volume 11/10. So this is my desperate plea... Can we PUHLEASE stop this AI Confidence Theater, people? It’s doing more harm than good.

Then immediately after:

This post is sponsored by Firecrawl. Firecrawl is the web data API to search, scrape, and interact with the web at scale. Turn the live web into clean, structured data your agents can actually use.

(P.S. I personally use Firecrawl in my Lovable apps all.the.time - most recently to scrape this very blog so my AI double could use my latest posts as context.)

7 bulan yang laludiscuss

Hey, I'm the founder of Datalab (we released Chandra OCR). I see someone requested it below - happy to help you all get setup. I'm [email protected]

7 bulan yang laludiscuss

Yes, we can sign a BAA!

7 bulan yang laludiscuss

Hi, I'm a founder of Datalab. I'm not trying to take away from the launch (congrats), just wanted to respond to the specific feedback.

I'm glad you found a solution that worked for you, but this is pretty surprising to hear - our new model, chandra, saturates handwriting-heavy benchmarks like this one - https://www.datalab.to/blog/saturating-the-olmocr-benchmark ,and our production models are more performant than OSS.

Did you test some time ago? We've made a bunch of updates in the last couple of months. Happy to issue some credits if you ever want to try again - [email protected].

tahun laludiscuss

I assume you're using a PDF, and not the image you shared? You need to set force ocr or format lines to get inline math with a PDF (for images, we just OCR everything anyways, so you don't need any settings).

We're working on improving the playground generally now - expect a big update tomorrow, which among other things will default to format lines.

Thanks for the kind words! The team was just me until pretty recently, but we're growing quickly and will be addressing a lot of issues quickly in the next few weeks.

tahun laludiscuss

Hi, author of marker here - I tried your image, and I don't see the issues you're describing with the newest version of marker (1.7.5).

I ran both with no setting specified, and with force_ocr, and I didn't see the issues either time.

tahun laludiscuss

Thanks for sharing! I'm training some models now that will hopefully improve this and more :)

tahun laludiscuss

Benchmarking is hard for markdown because of the slight formatting variations between different providers. With HTML, you can use something like TEDS (although there are issues with this, too), but with markdown, you don't have a great notion of structure, so you're left with edit distance.

I think blockwise edit distance is better than full page (find the ground truth blocks, then infer each block separately and compare), but many providers only do well on full pages, which doesn't make it fair.

There are a few different benchmark types in the marker repo:

  - Heuristic (edit distance by block with an ordering score)
  - LLM judging against a rubric
  - LLM win rate (compare two samples from different providers)

None of these are perfect, but LLM against a rubric has matched visual inspection the best so far.

I'll continue to iterate on the benchmarks. It may be possible to do a TEDS-like metric for markdown. Training a model on the output and then benchmarking could also be interesting, but it gets away from measuring pure extraction quality (the model benchmarking better is only somewhat correlated with better parse quality). I haven't seen any great benchmarking of markdown quality, even at research labs - it's an open problem.

tahun laludiscuss

I ran a partial benchmark against marker - https://github.com/VikParuchuri/marker .

Across 375 samples with LLM as a judge, mistral scores 4.32, and marker 4.41 . Marker can inference between 20 and 120 pages per second on an H100.

You can see the samples here - https://huggingface.co/datasets/datalab-to/marker_comparison... .

The code for the benchmark is here - https://github.com/VikParuchuri/marker/tree/master/benchmark... . Will run a full benchmark soon.

Mistral OCR is an impressive model, but OCR is a hard problem, and there is a significant risk of hallucinations/missing text with LLMs.

tahun laludiscuss

I'm a fan of the team of Allen AI and their work. Unfortunately, the benchmarking of olmocr against marker (https://github.com/VikParuchuri/marker) is quite flawed.

Throughput - they benchmarked marker API cost vs local inference cost for olmocr. In our testing, marker locally gets 20 - 120 pages per second on an H100 (without custom kernels, etc). Olmocr in our testing gets between .4 (unoptimized) and 4 (sglang) pages per second on the same machine.

Accuracy - their quality benchmarks are based on win rate with only 75 samples - which are different between each tool pair. The samples were filtered down from a set of ~2000 based on opaque criteria. They then asked researchers at Allen AI to judge which output was better. When we benchmarked with our existing set and LLM as a judge, we got a 56% win rate for marker across 1,107 documents. We had to filter out non-English docs, since olmocr is English-only (marker is not).

Hallucinations/other problems - we noticed a lot of missing text and hallucinations with olmocr in our benchmark set. You can see sample output and llm ratings here - https://huggingface.co/datasets/datalab-to/marker_benchmark_... .

You can see all benchmark code at https://github.com/VikParuchuri/marker/tree/master/benchmark... .

Happy to chat more with anyone at Allen AI who wants to discuss this. I think olmocr is a great contribution - happy to help you benchmark marker more fairly.

tahun laludiscuss

Docling is a great project, happy to see more people building in the space.

Marker output will be higher quality than docling output across most doc types, especially with the --use_llm flag. A few specific things we do differently:

  - We have hybrid mode with gemini that merges tables across pages, improves quality on forms, etc.
  - we run an ordering model, so ordering is better for docs where the PDF orde ris bad
  - OCR is a lot better, we train our own model, surya - https://github.com/VikParuchuri/surya
  - References and links
  - Better equation conversion (soon including inline)

tahun laludiscuss

Marker (https://www.github.com/VikParuchuri/marker) works kind of like this. It uses a layout model to identify blocks and processes each one separately. The internal format is a tree of blocks, which have arbitrary fields, but can all render to html. It can write out to json, html, or markdown.

I integrated gemini recently to improve accuracy in certain blocks like tables. (get initial text, then pass to gemini to refine) Marker alone works about as well as gemini alone, but together they benchmark much better.

tahun laludiscuss

It's on the list to build - been focusing on quality pretty heavily lately.

tahun laludiscuss

Hey, I'm the author of marker - thanks for sharing. Most of the processing time is model inference right now. I've been retraining some models lately onto new architectures to improve speed (layout, tables, LaTeX OCR).

We recently integrated gemini flash (via the --use_llm flag), which maybe moves us towards the "hybrid system" you mentioned. Hoping to add support for other APIs soon, but focusing on improving quality/speed now.

Happy to chat if anyone wants to talk about the difficulties of parsing PDFs, or has feedback - email in profile.

2 tahun yang laludiscuss

Datalab | NYC | Full-time | Software Engineer and Head of Business Ops | $250k-$350k + 1.5-3% equity | https://www.datalab.to

A significant % of useful data is locked away in tough-to-parse formats like PDFs. We build tools to extract it, like https://github.com/VikParuchuri/surya (15k Github stars), and https://github.com/VikParuchuri/marker (19k stars). We also run an inference API and product.

We do meaningful research (we’ve trained several SoTA models), ship product, and contribute to open source. We’re hiring for 2 roles to help us scale:

Senior fullstack software engineer

  - work across our open source repos, inference api, and frontend product
  - interact with our user community
  - you’ll need to be pragmatic, and embrace “boring” technology.  Our stack is fastapi, pytorch, htmx, postgres, and redis.  We deploy to render, and do inference with serverless gpus.
  - requires having built something impressive, ideally an open source project

Head of business operations

  - first non-technical hire
  - work across multiple areas, including finance, hiring, and sales
  - you’ll need to be extremely organized and able to get a lot done
  - requires experience leading operations at an early stage

Email [email protected] if you’re interested - include a link to something you’ve built if possible. You can also read more here - https://datalab-to.notion.site/careers-jan .

2 tahun yang laludiscuss

Hi, I'm the author of surya (https://github.com/VikParuchuri/surya) - working on improving speed and accuracy now. Happy to collaborate if you have specific page types it's not working on. For modern/clean documents it benchmarks very similarly to Google Cloud, but working on supporting older documents better now.

2 tahun yang laludiscuss

Hi, I'm the author of marker - https://github.com/VikParuchuri/marker - from my testing, marker handles almost all the issues you mentioned. The biggest issue (that I'm working on fixing right now) is formatting tables properly.

2 tahun yang laludiscuss

Working on improving tables soon (I'm the author of marker)

2 tahun yang laludiscuss

This isn't specifically tuned for tables (more for general pdf to markdown), but it's worked for some people with similar use-cases - https://github.com/VikParuchuri/marker

2 tahun yang laludiscuss

For PDF to markdown, I recently released V2 of my tool marker - https://github.com/vikparuchuri/marker