The combined professional network of founders and initial investors can minimize impact and effects of many factors discussed in other comments here.
"If you need to make a single cold call in the first 2-3 years, consider your network might be insufficient."
Such cold calls would include finding: co-founders, investors, customers, service providers, manufacturers, employees, etc.
Mind you-- I got this wrong in my own startup that didn't survive. Having advised founders and non-tech entrepreneurs in Seattle, SF and Vancouver and of those who heeded the advice, they succeeded where I didn't. E.g., when I had a co-founder flake on me, my network was insufficient to have another person I could trust to step in as subject matter advisor. Not that I would have expected that advisor to write code but perhaps to guide me through the principles of the niche topic such that I could then find the right contractor or employee.
Likewise, for some businesses including certain SaaS: picking your first customers carefully often requires a chain of trust in both directions; i.e., your network.
Have you considered using Common Crawl [1], and if so, what was your assessment when compared to having your own spyders?
Long-term, a combination of theirs and your own could be optimal.
There are strengths and weaknesses with using their dumps: on one hand, benefits include them having crawled and having dealt with being throttled, etc. They offer monthly dumps for general content and daily dumps for news [2].
On the other hand, it's a huge pile of data to wade through, and their index format might not be your preferred method. The archive and index reside officially at AWS, so that may decide where to process it. (Not sure whether other providers maintain a copy as well or not.)
By "huge", specifically:
> October 2020 [...] contains 2.71 billion web pages or 280 TiB of uncompressed content.
From our analysis a few years ago, that was to be the approach for the now-defunct Snagz.net [3] (which never fully launched because co-founders were unable to join due to extenuating circumstances).
There's a list of Common Lisp resources on Xach's blog [1] that includes communities, which remains relevant despite being an old post.
Also, contacting one of the people associated with a blog listed on Planet Lisp [2] would ensure that it appears there and might be seen by a wide yet relevant audience.
There are several topics intertwined with solutions you seek: There's parts of speech (PoS) tagging, reducing to Lemma form, identifying end of sentence, etc.
After having faced a similar learning curve, I put what I know into a lengthy document[0] written in 2018 based upon explorations over 2016-17. That will get you deployed and operational quickly by following just the final section. The first section explains key concepts using conventional ideas as the means of introducing NLP jargon. In between covers theory and practice for getting the most out of any tool you're likely to use in the end.
More general tools are probably available today, such as add-ons for Elasticsearch. I'd start looking there. Interesting items came up when searching ddg for: NLP elasticsearch.
Off-topic with respect to Kotlin but relevant to the Java platform:
From your perspective, what are Clojure's prospects for contemporary and future alignment with the JVM?
After all, Rich Hickey built the core in 2006-07. Looking at his graph[0] of code introduction and retention in his HOPL-IV paper[1] might imply that either he's that good at the core abstractions or possibly the language could suffer a similar set of limitations as commented about Kotlin above.
For others considering Rust + Kotlin for native Android development, there's also the Visly blog post[1] with basic steps. However, since then, Google officially released `androidx`. Unfortunately, Android Studio or the SDK lack `cargo fix` features of modifying your source code and instead just magically compiles with a compatibility layer. There's a PR on Visly's GitHub repo with these code fixes.
Also, their code base is minimal: does just enough with Rust shared library to have an app generate a log message. That's enough if you already know Android development.
For a complete end-to-end example search query screen with results screen (Activity, Fragment) using Rust, Kotlin, androidx, RecyclerView, ViewModel, etc., there's [2], but it was written while learning all things Android.
Edit: Yes, that second one is mine. Had fun learning but hopefully will never have to touch mobile dev again beyond maybe a PWA. (I'm a server-side dev.)
As someone just getting into Nix and NixOS myself, I found one particular comment there offers an excellent summary after talking with a few others in person as well:
https://news.ycombinator.com/item?id=22891648
Essentially, after an initial investment of time for learning curve, the huge payoff comes when migrating to another machine because of system-wide configuration being reproduced with precision.
In my case, moving back and forth between laptop and desktop/workstation was a pain point with Xubuntu. A delta of less than one hour between `apt-get dist-upgrade` on two devices was enough to translate into divergence of versions in some packages.
We've come a long way since days of tsx-11, but I digress.
Résumé/CV: Linkedin.com/in/dpezely or Pezely.com/daniel
Email: first name at last name dot com
Currently specializing in Rust:
- Over 2.5 years as primary language, and began with 1.0-stable in mid-2015
- Rust mixed with Kotlin for native Android app, where Kotlin handles just UI via RecyclerView, ViewModel, etc. (otherwise, I'm relatively new to mobile dev; Rust lib originally deployed in web app, and the plan is to use the same Rust library with Swift for iOS)
Generalist from prior specializations in concurrency, exotic data storage, dev-ops, Natural Language Processing
Samples:
- GitHub.com/dpezely/anagram-phrases (using prime number factorization, in Rust)
- GitLab.com/dpezely/chat-server (mio as the only networking library, in Rust)
Daniel Pezely — Authorized to work for any employer in US or Canada
Location: Vancouver
Remote: Negotiable, experienced
Willing to relocate: Negotiable
Technologies: Rust, Python, Cython, Common Lisp, Erlang, C; AWS, GCP; Debian-based Linux, FreeBSD, macOS/MacOSX; old school pure JavaScript in browser to exercise server-side API for delivering end-to-end functionality; learning Kotlin+Rust for Android Native
Résumé/CV: Linkedin.com/in/dpezely or Pezely.com/daniel
Email: first name at last name dot com
Currently specializing in Rust:
- Over 2.5 years as primary language, and began with 1.0-stable in mid-2015
- Rust mixed with Kotlin for going full Android Native, where Kotlin handles just UI (otherwise, I'm new to mobile dev)
Generalist from prior specializations in concurrency, scaling, exotic data storage, dev-ops, and long ago as a Unix sys-admin.
e.g., at Splunk/BugSense: rewrote the BugSense back-end from Erlang/Lisp/C to pure Erlang for ironically 25x performance increase. Server cluster saw several billion inbound requests per day non-stop from all time zones. (See GitHub for slides from Erlang Factory and other presentations.)
Samples:
- GitHub.com/dpezely/anagram-phrases (using prime number factorization, in Rust)
- GitLab.com/dpezely/chat-server (mio as only networking library, in Rust)
Indirectly, you may be able to accomplish tracking success by following the smart money, so to speak.
There's a hedge fund encouraging companies for which they have significant holding, that if these companies don't improve their carbon footprint and related impact, management would be voted out. (overly simplified)
Hopefully more people identifying themselves as software engineers will use Finite State Machines to better ensure correctness of the system.
Within a San Francisco company recently, suggesting that they create FSM diagrams was met with resistance that such things seemed frivolous.
Asking for Sequence Diagrams, they presented an architecture map.
This confusion occurred with most of the people I asked: e.g., team lead and manager, each with more than a decade of professional experience. This company is well over a decade old with significant recurring revenue, so obviously they must be doing everything right-- Right? Suggesting how to have assurances for correctness was eventually taken into consideration, so I commend them for that much.
Unfortunately, they are not unique in that way. While I only have a small sample size of a handful of Silicon Valley companies that I've experienced firsthand as employee or visitor, that was the pattern at all but one. (The one exception was BugSense, by the way: freshly imported from Greece.)
At least with a tool like sketch.systems, defining an explicit FSM as part of a prototyping tool may satisfy places that insist that "docs go out of date".
Suggestions for features:
1. in addition to the JavaScript for behaviour mock-up, consider also supporting WASM.
2. If the behaviour mock-up code could be called from an API, integration tests could also confirm a mismatch between design versus actual.
Also consider viabale options for Plan B: on the job training worked well for us even before Edition 2018 landed.
We had a senior level Java developer learn Rust on the job and be very productive within a couple of weeks. I joined shortly after and was experience with Rust. This former Java dev and I shared knowledge equally: I learned the Diesel ORM and our PostgreSQL integration from him with a few short conversations, and I perhaps helped abbreviate his remaining Rust learning curve with maybe three 1-2 minute conversations.
Another person at that same company was primarily a React developer, and he picked up Rust in fairly short order as well, but I had less involvement there.
A third person was fresh out of an extended bootcamp-style co-op program, and knew Java and React & React-Native, but this her first position after graduating. Note that she came from a non-programming background prior: goods trading industry. Also English is her third or fourth language. I walked her through potentially tricky bits of syntax and semantics (such as descructuring-bind for Structs) in an afternoon. Edition 2018 had long-since landed by her arrival, so that helped.
Modifying existing Rust code and following the patterns worked well. She started small but progressed quickly. Within a few short months, she was adding back-end API endpoints (actix-web pre-1.0) for her front-end needs, and she was creating new .rs files with most PRs merged as-is.
From prior experiences, such as using Python from 1999 through 2005 and Common Lisp between 2005 through 2012, you have an advantage when recruiting for a language considered "exotic" by standards of the given day.
At two different startups circa 2007 and early 2008, we received many, many very high quality candidates with a one sentence description posted to the right place (e.g., planet.lisp.org) saying, "Use Common Lisp in Seattle."
I've had comparable experience with that approach for Rust in Vancouver, BC last year when looking for devs that were merely "familiar" with the language, even if no professional work experience with Rust but "Ideal Candidates" having "Rust, Go, C++, C, Python or Lisp family of languages". That was posted via this-week-in-rust.org.
That one was posted with relatively poor posted salary, yet we still saw high quality candidates for the senior level role. The intermediate level dev that was ultimately hired apparently worked very well for the company after I left.
Thanks to the author and contributors for putting in the time and effort to create this.
Especially the cross-comparison in the second half is very thorough and useful!
Perhaps this or something similar could be hosted on a public git repo so that it can be maintained by a wider set of contributors?
There was a similar survey[0] of Common Lisp systems from 2007-2010 from before services like Github/Gitlab became prevalent, so maybe we can have something like this Rust library survey NOT suffer the same fate. (Yes, Fare maintained it for a while--thanks, Fare!-- after Dan passed away too young, and then it became less detailed[1], but you get the idea...)
The overhead of C++ adoption actually prevented us from ever moving fully in this direction.
Today, unlike in 2013 when they started, there are other options.
For others considering code-sharing, another option not mentioned in their article is Rust for the core with Swift and Kotlin. Between bindgen, futures, serde_json and non-nullable pointers in Rust, those would satisfy their stated subcategories today.
Companion templates are explained here: (January 2019)
Medium.com/visly/rust-on-ios-39f799b3c1dd
Medium.com/visly/rust-on-android-19f34a2fb43
(No affiliation other than starting to proceed down this path myself.)
For Common Lisp heads, there's also MOCL, which seemed quite reasonable when I explored it years ago: wukix.com/mocl
Details may change every few years but overall, it boils down to two main approaches: 1) find a position through your professional network, as an estimated 80% of jobs come through such connections; 2) be discoverable for the other 20%.
Be active in both.
Present something at a local meeting/event. Often, it doesn't have to be fancy or overly polished. Some events on meetups.com have a few people present for 10 minutes each.
Which leads to #2, be discoverable: put those presentation materials on GitHub or GitLab.
Additionally, there are common interview questions (whiteboard and/or code challenge) such as URL shortener (e.g., TinyURL) or anagram solvers. These have come up a number of times in my interviews over the past few years, so I wrote the code and pinned these on my GitHub profile. (Same user id as here, if you're interested.)
I decided to go a step further and give each the full dev-ops treatment: Dockerfile, Makefile, etc.
But give them a solution, not a problem.
Avoid relying upon AWS/GCP/Azure proprietary offerings unless that's precisely what you're demonstrating, and make it work as-is: fetch all dependencies with version pinning (if possible-- else put versions in README), so then you know that it will build for them exactly as it did for you.
Some differences between then and now are also different idioms such as CSS naming conventions.
See GetBEM.com - for Block, Element, Modifier naming.
For instance, current conventions are to put a class name on far more HTML elements than years earlier, and avoid using hierarchical specifiers. (Yes, I know, at first it may seem to defeat the cascading within "CSS" without the hierarchy, but it's more nuanced than first glance.) It's especially useful when hiring front-end/UI developers to maintain the project after your initial build, as many will have learned BEM at various coding bootcamps.
Daniel Pezely - Authorized to work for any employer in US or Canada
Location: Vancouver, BC
Remote: Yes
Willing to relocate: Negotiable
Technologies: Rust, Python, Common Lisp, Erlang, C; GCP, AWS; Debian-based Linux, FreeBSD; old school pure JavaScript in browser to exercise server-side API for delivering end-to-end functionality
Résumé/CV: Linkedin.com/in/dpezely or Pezely.com/daniel
Email: first name at last name dot com
Currently specializing in Rust: 2+ years as primary language, began with 1.0-stable in mid-2015.
Generalist from prior specialization of concurrency, scaling, exotic data storage, dev-ops and long ago as a Unix sys-admin.
e.g., at Splunk/BugSense: rewrote the BugSense back-end from Erlang/Lisp/C to pure Erlang for (ironically) 25x performance increase Server cluster saw several billion inbound requests per day non-stop from all time zones. (See GitHub for slides from Erlang Factory and other presentations.)
Recently added:
GitHub.com/dpezely/anagram-phrases (using prime number factorization in Rust)
A utility for transforming JSON at scale written using the Rust programming language is bumblebee and available on crates.io.
The author of that package uses it internally for massive amounts of data on a regular basis, and he's a bit too modest to use a higher version number.
"If you need to make a single cold call in the first 2-3 years, consider your network might be insufficient."
Such cold calls would include finding: co-founders, investors, customers, service providers, manufacturers, employees, etc.
Mind you-- I got this wrong in my own startup that didn't survive. Having advised founders and non-tech entrepreneurs in Seattle, SF and Vancouver and of those who heeded the advice, they succeeded where I didn't. E.g., when I had a co-founder flake on me, my network was insufficient to have another person I could trust to step in as subject matter advisor. Not that I would have expected that advisor to write code but perhaps to guide me through the principles of the niche topic such that I could then find the right contractor or employee.
Likewise, for some businesses including certain SaaS: picking your first customers carefully often requires a chain of trust in both directions; i.e., your network.