With OpenAI and other LLMs, web development is accelerating. For example, I put together a AI call center demo ( https://www.youtube.com/watch?v=Vv7mI_qRrhE ) by using Open AI o1-preview. There I could take a lot of different files on typescript and backend server stuff written in python. I would add logs into the mix to make one massive prompt, and then I would let the AI work on reasoning in the cases where I needed to accelerate the writing of additional code.
Building on this, Human preference optimization (such as Direct Preference Optimization or Kahneman Tversky Optimization) could be used to help in refining models to create better data.
If you are using no-code solutions, increasing an "idea" in a dataset will make that idea more likely to appear.
If you are fine-tuning your own LLM, there are other ways to get your idea to appear. In the literature this is sometimes called RLHF or preference optimization, and here are a few approaches:
Direct Preference Optimization
This uses Elo-scores to learn pairwise preferences. Elo is used in chess and basketball to rank individuals who compete in pairs.
@argilla_io on X.com has been doing some work in evaluating DPO.
KTO is an approach that uses mono preference data. For example, it asks if a response is "good or not." This is helpful for a lot of real word situations (e.g. "Is the restaurant well liked?").
Data scientist here who spent a couple of years working with Unreal (to produce high end data visualizations). Here are my thoughts
> Blueprints suck!
Not really. Think of Blueprints like python. Its good for routing and keeping track of things at a high level. Think of C++ as handling things at a lower level.
> I heard you need to start with blueprints.
Not really. After going through the basic tutorial that Unreal Sensei has on YouTube (https://youtu.be/gQmiqmxJMtA?si=TqBiiIe12M5hiCda) , it is better to do a mix of blueprints and C++ if you have any programming background.
> I don't know what to use for the IDE.
I used Rider for Unreal Engine and it has good integration into Unreal Engine.
> So when do you use C++?
When I was doing data vis of census data, I needed a way to load in 10,000 data points into memory. The "out of the box" tools for Unreal didn't support this, so custom C++ was the way to go.
> But really, if I just want to get started with Unreal and want official tutorials, where do I go?
Also, a year ago I put together an online course that looked at how to ramp up on Unreal Engine. The course ("Data Visualization in the Metaverse") is ideal if you already have a programming background. I put the course out on YouTube for free (https://www.youtube.com/playlist?list=PLKH3Xg62luIgPaB4fiFuT...) and happy to answer any questions about it.
10 thoughts on data visualization best practices and tools:
1) For interactive visualizations of data on 3D globes, I use a mix of C++, Python (for data cleaning), and Unreal Engine (with a plugin called Cesium). An example of this is at https://youtu.be/9i-tQ8Sr80o.
4) If you are trying to do 3D population density maps in R, there are a lot in the community that say you should use https://www.rayshader.com/ with R.
6) If you are doing data vis for urban planning, odds are they are already using ArcGIS, and odds are you will be using something like that.
7) If you are trying to do data vis that relates to architecture, I would actually suggest starting with Twinmotion (which is part of the Unreal Engine ecosystem).
9) If you are wanting to show some high end maps fast, use Geolayers 3. There is a YouTube channel called "Boone Loves Video" (https://www.youtube.com/channel/UCXyGw2OkrAzLhq1r7hyDZkA). Boone explains Geolayers often in his videos.
10) I personally believe that if you are trying to get to next-gen data visualization my best guess is that you would use a mix of Blender, Nuke, Houdini, or After Effects. I personally have only used Blender and After Effects so far.
Building on this, for many companies, the leader of the IT org has the main responsibility of focusing on end (or outside) customer needs at the highest quality with a low "Total Cost of Ownership".
I just put together a tutorial for data analysts who want to learn BigQuery. The tutorial goes through the process that I use as a data scientist to examine data.
For the tutorial, I took a look at election contributions based on Federal Election Commission data. The numbers seem to align with what is reported in the press (so the analysis seems sound), but feedback is always welcome.
OP Here. When I started out, this blog was going to be one paragraph and it turned into data journalism that has an interactive 3D globe which shows housing prices, and interactive charts that refine data. Along the way, I refined the blog so that it also teaches data analysts how to use real, recent real estate data to create their own charts in Python. Happy to answer any questions about what I put together.
The short answer is you want to learn a language that helps you achieve your short-term goals and that ideally positions you to achieve your long term goals. Focus more on the goal, and select a popular tool (so you get support) that has a good chance of solving your business problem.
To give some perspective, in a 22 year career, I started out learning Visual Basic, C++, SQL, Java, and databases because those were “hot at the time”, and it was fairly easy to get jobs building out apps with these kind of technologies.
As the web started to gain momentum, I found that again I chased short term goals of doing web development by working with HTML, CSS, and JavaScript.
After business school, I started to refocus on longer term opportunities. I learned about predictive models, R, Python, Docker, Hadoop, Spark and TensorFlow and applied all of that to data science and machine learning engineering.
These days, I am building a business that teaches people what I have learned over the years with current technology. So here, I have come full circle and have done a lot of React, Node, and Tailwind work to build a website (https://www.whiteowleducation.com) that teaches people how to data visualizations using Python, C++, and Unreal Engine.
It seems like it rate hikes would reduce demand, and I think policymakers believe a reduction in demand will force the price reductions.
For example, in the US, recent interest rate hikes led to even larger mortgage rate hikes. The increase in mortgage rates had the effect of making it more difficult for people to afford homes. Once mortgage rates started rising here, I started to see a reduction in homes sold and it has started to lead to a slight reduction in home value.
I put together a markdown outline for new features to implement and new marketing ideas. I also put together an outline for my current project.
Before starting a current project, I spend a couple of weeks really thinking through the value that will be created for humanity if the project is completed. Each time a new idea comes up, I put it in the outlines above, but I make it a point to COMPLETE the current project by telling myself about all of the end value that will be created.
When I am coding, I feel the need to "type it out."
If I am writing blogs or if I'm writing scripts, I find that I can dictate much faster and then just edit the sentences afterwards in order to clarify. The process of dictation plus editing is faster for me than if I were to type out everything.
Mavis Beacon taught my how to type between 60-75 wpm. The dictate feature of Microsoft Word within Office 365 gets me roughly to 100 wpm after corrections.
Bottom line: Mavis Beacon helped me code faster, but Microsoft helped me write blogs faster.
I feel the best strategy to win at Wordle would be to see if someone already posted the answer.
If no answer, use reliable hints ( i.e on Twitter) to better inform the initial guess.
Beyond that, the best initial word goes beyond information theory, letter frequency, etc to also be a word that has a realistic chance of being word of the day. As a trivial example, AEIOU may reveal some information on the first guess, but I find it extremely unlikely that you will win Wordle in 1 with this guess.
Is there a reasonably priced service (for individual budgets and not corporation budgets) that would allow you to rent a satellite in order to for example count the number of Teslas that are out on the road in a given city in the United States?