kxyvr · HackerTrans

kxyvr

1,286 karmajoined 12 yıl önce

Submissions

Show HN: Optizelle, a library for nonlinear optimization

github.com

2 pointsby kxyvr10 yıl önce0 comments

comments

11 gün öncediscuss

Money absolutely has an impact across easily available metrics. For example, money lets one purchase better coaching for sports or join a travel team. As we see from the lawsuit, athletes were accepted at a much higher rate than the rest of the applicants. Money lets one attend better schools either from a transfer or private school. Money allows for better tutoring. I completely agree.

Where I think we differ is that my contention is that money also allows one to game a higher test score on something like the SAT and by a significant factor. Yes, it would focus attention on this one factor, but I believe it also misses the point. College admissions is not meant to be a singular contest or game, but part of a broader ecosystem and that system has choices.

A university has a number of competing criteria and wants for its applicants. Generally speaking, a university does not want someone to drop out. They want graduates and part of screening applications is meant to determine whether they believe someone can successfully complete their degree. Some universities essentially operate like hedge funds that have classes. These universities have an interest in high net worth and connected applicants because these people can trade favors or give donations as alumnae. This is reflected in legacy admissions and automatic admission for certain classes of donors. Some universities want to give preference to in-state or in-city students. The purpose here is to ensure the university is locally connected because the state and city have the most influence on what is really a city inside of a city for most universities. Some universities want to give children of those who never went to college preference because it ultimately broadens their base and likely ensures their children will want to attend there as well. Some universities want to maximize their rating on the US News and World Report ranking because it means they can charge higher tuition.

None of those metrics above, moral or amoral alike, need a total ordering of applicants based on a test score.

As one other comment, when I sat on the graduate board, our department was flooded with applications from one particular country, more than half. Don't know why, but it was. Certainly, we accepted some of those applicants. However, consider the implications of accepting students purely from that one place. Would the local government be happy if we did not accept any local or domestic students? Would the federal? Probably not. However, if we accepted applications purely based on a singular test score, there would be incentive for that place to game the system and help with the test, legal or otherwise. No idea if they did then, but it wouldn't have mattered since we still read the broader application.

11 gün öncediscuss

My continued assertion is that "hard data" can be misleading, wrong, and discriminatory. It requires judgement to understand what it tells us and whether there is error in the collection of the data itself.

As far as Kaplan, I don't have metrics to share, but what I saw shocked me at the time. It appeared as though they collected data as to how much their test prep improved scores and it was pretty dramatic. And, they were smart enough to distinguish between their test prep books and their in person tutoring. In person tutoring worked better and by a lot, but it all helped.

As far as tests, I've never contended that they shouldn't be done, but that these scores need to be part of an overall portfolio. I have in fact sat on admittance boards for graduate school and the two dominant factors other reviewers used were GPA and where they went to undergrad. Both of these were hard metrics and full of bias. I'll also contend that they weren't particularly effective in identifying students who successfully completed the program.

Something to consider, I do think test scores can help screen candidates. For example, if one is going to admit into a math program, then their quantitative score should probably be above a certain level. After that, I personally don't think it matters. In that way, one could just set minimum criteria to be accepted, which includes GPA, tests scores, research, jobs, whatever, and then randomly select from that group. That would certainly eliminate a huge amount bias, though not all, but most people hate this approach because it contradicts this belief that we live mostly in a meritocracy. They want rankings and they want to know who number 1 is. My continued argument is that it's impossible to distill a person down to a hard ranking in a reasonable matter, especially with test scores, and it's not necessary for college admission since one can select a group of applicants for acceptance without ordering them.

11 gün öncediscuss

Thankfully, I also work in the adult world and this world requires judgement.

As an example, my wife is a physician. She and her colleagues use metrics to help assess a patient, but ultimately treatment is based on their judgement. Purely relying on hard numbers, such as what an EKG gives, would lead to dead patients. Many of them. I am a mathematician. My world revolves around hard computational numbers, yet my algorithms will often given misleading results. Determining when that is requires judgement. Despite a push too the contrary, good hiring managers assess applicants and take responsibility for both good and bad hires. This requires judgement.

And, yes, I can also find examples where someone either didn't study or used a book and did well on their SAT. That said, in person test prep with a tutor absolutely has a large, positive effect on test scores, much more than simply a book. Kaplan has the internal data. It's absolutely worth the money if one can afford it. Most can't.

11 gün öncediscuss

I still contend strongly that it is wrong to distill a person into a collection of numbers. They are not.

My mother was a special ed teacher who eventually became a diagnostician whose job it was to determine eligibility for services. Part of this job was to give a standardized test to a child to see if they met state set criteria. That said, this was only part of an assessment and was used along with parent preference, teacher recommendations, and other people involved with the process at the school. Outside of meeting state criteria, the numbers in the test were used to identify areas of strength and weakness to help understand a child and put this in perspective of a broader story. For example, if a child struggled with math, was it because they couldn't read well, struggled with the math itself, was distractable? There were also other instances where the test scores themselves were misleading. For example, did a child perform poorly because they were sick, or the AC of the building broke, or they set a test date on a school holiday when the child knew the other kids had a day off?

My point in telling this story is that the raw score in a standarized test can be helpful, but it does not tell a complete story. It is part of a broader assessment. The SAT and college admission is not different.

Purely using a test score does provide an objective score to rank a collection of applicants, but I still contend strongly that it misses the point. Depending on your point of view, college is meant to educate a populace, provide job training, give economic opportunity, or a whole host of other goals. One can use test scores a proxy to determine which applicants best approximate these goals, but I think we should consider whether they really help achieve this.

I'll also mention that using tests scores is not entirely "fair" either. I'd a friend who dated a guy who worked for Kaplan, one of the test prep companies. They had very good data about how much their test prep improved test scores. They were also very expensive. As a result, money and wealth can be used pretty directly to improve one's score. It's difficult for a university to fully address wealth inequality in society. However, since tests can be gamed so easily, I would caution against their over use.

11 gün öncediscuss

There's a wonderful exchange in the movie Interstellar that speaks to this:

    Cooper: You're ruling my son out for college now? The kid's fifteen.
    Principal: Tom's score simply isn't high enough.
    Cooper: What's your waistline? 32? With, what, a 33 inseam?
    Principal: I'm not sure I see what you're getting at.
    Cooper: You're telling me it takes two numbers to measure your own ass but only one to measure my son's future?

The point being is that a person, and their future, should not be distilled into a single number like an SAT score because people are far more complex than a single number. I would also contend that splitting them into a few numbers, such as by subject area, doesn't help either.

5 ay öncediscuss

It's a combination of factors. Rural hospitals and clinics tend to be under-resourced with lack of equipment in buildings that aren't particularly nice. As far as small town, if you like it, great. However, people who are highly educated tend to like to be around others who are similarly educated and that's difficult to find in a rural town unless it's also a university town. There tends to be a lack of school options for their children and given how much they spent on their own education, they tend to prioritize this highly. There tends to be a lack of town infrastructure like good grocery stores, or theater, or museums, or other amenities. Docs also have their own medical needs and understand that those can't be met at small clinics, so they like to have access to good hospitals. Imagine intimately knowing all the ways something like childbirth can kill you and also knowing that there's not an appropriately trained surgeon in town. By the time one finishes their training, they're probably in their 30s and may want to find a partner. Options tend to be limited in small towns. On the darker side of things, foreign people are often not particularly welcomed in rural towns and this can be a particularly bitter experience for the foreign docs that are essentially forced to work there.

So, no, it's not just an urbanite out of their comfort zone. There's a whole host of issues. And, to be clear, we need people to work these jobs, but it's not particularly pleasant for a lot of them.

5 ay öncediscuss

There's a video as to how the match works here:

https://www.nrmp.org/intro-to-the-match/how-matching-algorit...

Basically, you interview at a bunch of programs and then rank them. The programs (hospitals) rank applicants and then the algorithm does its magic to "match" applicants to programs. Now, if one doesn't match with any of them, there's something called the scramble where a med student works with their program to match into a program somewhere in some specialty that has room. This is non-ideal, but can work out.

Generally speaking, the match algorithm is setup to guarantee all U.S. medical school graduates a match somewhere in something. In may not be what you want, but you will have a job. Then, preference is given to things like the island schools (affiliated medical schools in the Carribean, which are very expensive, but somewhat easier to get into), and then to other international medical schools. Somewhere in there are also foreign physicians who want to work in the U.S., but are forced to redo residency.

I don't know everything about how it works, but that's the general idea. To that end, I don't fully understand the stats you pulled from the reference. That doesn't mean they're not valid, but I don't know.

And, yes, often times, there are open slots at some program in the middle of nowhere. As much as there can be incentives such some debt relief by working in rural hospitals, the jobs are not a good fit for a lot (most) people. I mean, someone just worked extremely hard for 10 years or more and you want them to go live in a town of 10k people. It's not that it's not important, but you can't force people to do it and it takes a particular personality to be happy there. A lot of highly educated people want to live in urban centers with amenities. Not all, but probably most.

Places like Canada use their foreign docs to fill this rural gap. A not small number of the rural docs are foreign born and trained and they essentially work this crappy jobs until they have permanent residency and then they move to more desirable markets. It's a trade, I guess, but there's not a small amount of resentment about it.

5 ay öncediscuss

If they don't match, they're allowed to scramble and move into one of those programs with open positions. If they don't choose to, that's on them, but it's still not a problem with number of residency slots.

I very much agree that pay is a barrier to entering specialties like family medicine. Though it depends on the market, I normally see family medicine at around $200k/year and that's not great if one needs to take something like $750k debt to get there along with eight years of training after a bachelors. If we want to fix that, then we need to make the value proposition better and reduce the medical school debt, improve working conditions, and/or increase pay.

So, yes, if one wants to maximize their earning potential, then they need to enter one of the specialty residencies and fellowships. Those are currently filled. However, that's not where the biggest need is and I contend that's not why there's a physician shortage.

5 ay öncediscuss

That's not true. You can look at the residency match for 2025 here:

https://www.nrmp.org/match-data/2025/05/results-and-data-202...

While many specialties are fully filled, we need pediatricians, family medicine, and internal medicine docs. They're generalists and where the largest shortage is. There were 147 unfilled slots for pediatricians, 805 for family medicine, and 357 for internal medicine. They don't have the applicants; it's not the slots.

5 ay öncediscuss

No, they are not paid too much. There's a lot of incorrect assertions here, so it'll take a lot to work through them.

Physician pay depends on specialty, but it can range from the low $100kish mark for pediatricians to $500-750k for certain kinds of surgeons. Family medicine tends to be around $200k. However, this amount ranges vastly by market and top pay often goes to those willing to work in more rural hospitals because no one wants to. For example, pay in NYC for physicians is appalling low compared to the rest of the U.S. market. In addition, certain systems have hard caps. For example, the VA hospitals cap physician pay inclusive of bonus at $400k. This is documented and you can in fact just look up a random doc at the VA with one of the many federal pay search tools.

While some doctors can make more, it typically because they own a practice and that increased pay comes from good old fashioned capitalism. Meaning, they tax the amount their nurses, NPs, medical assistants, etc. make just like all businesses make money per head on their employees. Whether you believe this is right or wrong is up to you. However, this is not any different that someone who runs, for example, a yard care business. More accurate pay can be found by those who work directly for large hospitals.

Next, the cost of medical education in the United States is vastly higher than other countries. Right now, medical school will cost you somewhere from $400-600k. This is in addition to whatever debt accrued during undergraduate. Further, medical school applications are highly competitive, so students often accrue additional debt by completing a masters in something like public health prior to entry to medical school. This means that someone may have upwards of $750k of debt when they finish medical school, but they still have somewhere between 3-10 years of residency and fellowship before they make attending money. During this time, the debt accrues interest and balloons.

Now, once you become an attending, you're still not good and expenses are vast. Shift work can vary from something like 7 12-hour shifts in a row for intensivists to 14 shifts in a row for hospitalists. Note, just because it says its a 12-hour shift doesn't mean you work 12 hours. They still need to chart and bill and if it's busy, that may be another few hours after the shift is over. In some remote clinics, an ER physican may work 7 24-hour shifts in a row. That may sound absurd and unsafe and it likely is, but it's the reality of the work. If someone is working that schedule, they have increased expenses to just, frankly, live. On the low end, it's very difficult to cook in that environment, so you have to buy a lot of premade food. On a more expensive end, having children on this schedule is extremely difficult. You either require a spouse that doesn't work or you need something like a night nanny. If you're working 12 hour shifts, you must sleep at night and you can't be up to take care of a baby otherwise you run the risk of killing someone the next day. Unless you're paying someone under the table, current nanny rates in large markets are about $20-25/hour. Insurance rates are also high. I don't mean malpractice either. Generally speaking, one needs to carry disability insurance because if one gets into a car accident and breaks their magic hands, there's no way to pay back that debt otherwise. These policies are thousands a year. That's just the start. They pay a large amount of money to buy their time back because they don't have it.

Next, there's a myth about limiting residency slots in order to increase pay, at least recently. I will not defend the AMA and some of they took, especially in the 1990s. Here's the 2025 residency match data:

https://www.nrmp.org/match-data/2025/05/results-and-data-202...

The number of offered and filled slots is on page 2 (or 13 depending on how you count). Some specialties filled all of their slots. Where the U.S. vastly lacks is pediatricians, family medicine, internal medicine (who can work like family medicine if need be.) Pediatricians had 147 unfilled slots. Family medicine had 805 unfilled spots. Internal medicine had 357 unfilled spots. These spots can be filled by people who graduated from U.S. medical schools, island medical schools, Mexican medical schools, or a vast array of other foreign medical schools. However, they're not filled because they don't have the applicants. That's not medical school collusion. That's the hard reality that medical school is extremely expensive and the training is extremely long.

Now, how do other countries handle things? One, their medical school is not as crushingly expensive. Two, places like Europe cap the number of hours a physican can work. If you want to pay American physicians less, you'd need to blow out their medical school debt, reduce their hours, and offer better benefits. Until then, no, really, they're not overpaid.

If you want to start pointing fingers, try the vertical integration of insurance companies, pharmacy benefit managers, and hospitals. I don't have the numbers readily available, so I'll stop here. But, really, it's not the docs.

10 ay öncediscuss

I'm an applied mathematician and this is the most common layout for dense matrices due to BLAS and LAPACK. Note, many of these routines have a flag to denote when working with a transpose, which can be used to cheat a different memory layout in a pinch. There are also parameters for increments in memory, which can help when computing across a row as opposed to down a column, which can also be co-opted. Unless there's a reason not to, I personally default to column major ordering for all matrices and tensors and use explicit indexing functions, which tends to avoid headaches since my codes are consistent with most others.

Abstractly, there's no such thing as memory layout, so it doesn't matter for things like proofs, normally.

geçen yıldiscuss

Automatic differentiation was actively and continuously used in some communities for the last 40 years. Louis Rall has an entire book about it published in 1981. One of the more popular books on AD written by Griewank was published in 2000. I learned about it in university in the early 2000s. I do agree that the technology was not as well used as it should have been until more recently, but the technology was well known within numerical math world and used continuously over the years.

geçen yıldiscuss

That's not true. Here's an abbreviated list from:

http://historyguy.com/major_wars_19th_century.htm

I'm sure there are others. It lists:

  Greek War of Independence (1821-1832)
  French invasion of Spain (1823)
  Russo-Persian War (1826-1828)
  Russo-Turkish War (1828-1829)
  Hungarian Revolution and War of Independence (1848-1849)
  First Schleswig War (1848-1851)
  Wars of Italian Independence (1848–1866)
  Crimean War (1854–1856)
  Second Schleswig War (1864)
  Austro-Prussian War (1866)
  Franco-Prussian War (1870-1871)
  Russo–Turkish War (1877–1878)
  Serbo-Bulgarian War (1885)
  Greco–Turkish War (1897)

Together, that adds up to multiple decades of war.

geçen yıldiscuss

True. That said, I'll also mention that tomography is a very rich, interesting field that's still open to new innovations. I work in the area and unfortunately needed to pass on a muon tomography contract some years ago. By the way, you may know this, but the following is for the broader audience.

---

If anyone is interested, the book Parameter Estimation and Inverse Problems by Aster, Borchers, and Thurber give an easy introduction to simple tomography problems in their book. Example 1.12 in their second edition has a very basic setup. More broadly, tomography intersects with an area of study called PDE constrained optimization. Commonly, tomography problems are setup as a large optimization problem where the difference between experimental data and the output of a simulation are minimized. Generally, the simulation is parameterized on the material properties of whatever is under study and are the optimization variables. The idea is that whatever material property that produces a simulation that matches the experimental data is probably what's there. This material property could be something simple like density or something more complicated like a full elasticity tensor.

What makes this difficult, is that most good simulations come from a system of differential equations, which are infinite dimensional and not suitable for running directly in an optimization algorithm. As such, care must be taken into discretizing the system carefully, so that the optimization tool produces something reasonable and physical. Words you'll see are things like discretize-then-optimize or optimize-then-discretize. Generally speaking, the whole system works very, very poorly if one just takes an existing simulator and slaps an optimizer on it. Care must be taken to do it right.

As far as the optimizer, the scale is pretty huge. It's common to see hundreds of millions of variables if not more. In addition, the models normally need to be bounded, so there are inequalities that must be respected. For example, if something like a density isn't bounded to be positive (which is physical), then the simulator itself may diverge (a simulator here may be something like a Runge-Kutta method.)

Anyway, it's a big combination of numerical PDEs, optimization, HPC, and other tools just to get a chance to run something. Something like the detector in the article is very cool because it may be a realistic way to get data to test against for super cheap.

2 yıl öncediscuss

I believe Absil, Mahony, and Sepulchre also have a book on optimization over manifolds:

https://press.princeton.edu/absil

I was unaware of the Bournal work, so thanks for that. Do you have any idea how Bournal's approach differs from Absil's?

For others, it looks like Bournal also has a book on the topic from 2023:

https://www.nicolasboumal.net/book/

2 yıl öncediscuss

In the U.S., there is typically a separation between calculus and real analysis. Though, the amount of difference between the two depends on the university.

In calculus, there is more emphasis on learning how to mechanically manipulate derivatives and integrals and their use in science and engineering. While this includes some instruction on proving results necessary for formally defining derivatives and integrals, it is generally not the primary focus. Meaning, things like limits will be explained and then used to construct derivatives and integrals, but the construction of the reals is less common in this course. Commonly, calculus 1 focuses more on derivatives, 2 on integrals, and 3 on multivariable. However, to be clear, there is a huge variety in what is taught in calculus and how proof based it is. It depends on the department.

Real analysis focuses purely on proving the results used in calculus classes and would include a discussion on the construction of the reals. A typical book for this would be something like Principles of Mathematical Analysis by Rudin.

I'm not writing this because I don't think you don't know what these topics are, but to help explain some of the differences between the U.S. and elsewhere. I've worked at universities both in the U.S. and in Europe and it's always a bit different. As to why or what's better, no idea. But, now you know.

Side note, the U.S. also has a separate degree for math education, which I've not seen elsewhere. No idea why, but it also surprised me when I found out.

2 yıl öncediscuss

Laugh. Probably! I gave a talk at that conference titled, "Software Abstractions for Matrix-Free PDE Optimization with Cone Constraints." I still work in the field, so you want to talk algorithms sometime, feel free to send me an email. I keep my email off of HN to limit spam, but if you search for the lead author on that presentation, it should list my website.

2 yıl öncediscuss

Honestly, the parent is pretty accurate. No one is claiming that P = NP. However, the technology to solved mixed integer programs has improved dramatically over the last 30 years and that improvement in performance has outpaced computational speed by multiple orders of magnitude. It's the algorithms.

I just went to pull up some numbers. The following comes from a talk that Bob Bixby gave at ISMP in 2012. He is the original author of CPLEX and one of the current owners of Gurobi. Between 1991 and 2007, CPLEX achieved a 29530x speedup in their solvers. Their 1997/98 breakthrough year attributes the following speedups, Cutting planes: 33.3x, presolve: 7.7x, variable selection: 2.7x, node presolve 1.3x, heuristics: 1.1x, dive probing 1.1x. I don't have a paper reference for these numbers and I don't think he has published them, but I was at the talk.

The point is that integer programming solvers perform unreasonably well. There is theory as to why. Yes, there is still a lot of searching. However, search in-and-of-itself is not sufficient to solve the problems that we regularly solve now. Further, that increase in performance is not just heuristics.

2 yıl öncediscuss

I'll second this. Their methods are very powerful and very fast. For those out of the loop, the Chebyshev (and ultra-spherical) machinery allows a very accurate (machine precision) approximation to most functions to be computed very quickly. Then, this representation can be manipulated more easily. This enables a variety of methods such as finding the solution to differential algebraic equations to machine precision or finding the global min/max of a 1-D function.

I believe they use a different algorithm now, but the basic methodology that used to be used by Chebfun can be found in the book Spectral Methods in Matlab by Trefethen. Look at chapter 6. The newer methodology with ultraspherical functions can be found in a SIAM review paper titled, "A Fast and Well-Conditioned Spectral Method," by Olver and Townsend.

2 yıl öncediscuss

A convex function is a function that is bowl shaped such a parabola, `x^2`. If you take two points and connect them with a straight line, then Jensen's inequality tells you that the function lies below this straight line. Basically, `f(cx+(1-c)y) <= c f(x) + (1-c) f(y)` for `0<=c<=1`. The expression `cx+(1-c)y` provides a way to move between a point `x` and a point `y`. The expression on the left of the inequality is the evaluation of the function along this line. The expression on the right is the straight line connecting the two points.

There are a bunch of generalizations to this. It works for any convex combination of points. A convex combination of points is a weighted sum of points where the weights are positive and add to 1. If one is careful, eventually this can become an infinite convex combination of points, which means that the inequality holds with integrals.

In my opinion, the wiki article is not well written.