orware

Should have checked before submitting. Seems like it was posted yesterday already: https://news.ycombinator.com/item?id=34526436

orware

4년 전discuss

I'd be curious to hear about experiences with persistent pooling usage within PHP, since I don't feel that I've heard about it being advised as much myself over the years, but that could perhaps be due to old misperceptions about it.

I know there is the "Persistent Database Connections" section of the PHP manual and the mysqli extension within PHP supports connection pooling / persistent connections, but in my own experiences I've rarely seen them utilized, especially by the bigger open source projects out there such as WordPress, which has an 8 year old enhancement topic on the subject: https://core.trac.wordpress.org/ticket/31018. Putting your database behind a pooler, like ProxySQL let's say, is another option as the level of sophistication for a company/application increases, but most typical PHP setups I've used don't have that immediately available.

I've generally been under the impression that most projects/applications don't use the built-in pooling features for some of the reasons discussed in the link above, leading to those applications being more impacted by lengthier connection times due to a new connection being created at the beginning each request and then closed at the end of the request.

Now I'm inclined to experiment a bit with the built-in mysqli pooling feature though since it would seem a worthwhile feature for developers to experiment with more if it would lessen the connection time impact for each PHP request, particularly for databases that are further away and require secure connections.

Shaving off 100ms for a connection would be significant for most PHP users if they are currently having to open fresh connections on each request, especially if they were previously used to connection times of < 1ms when connecting to a local MySQL database.

orware

4년 전discuss

I'm not an expert on the TLS 1.3 but the 0-RTT feature seemed like it wasn't implemented by a lot of clients so the new QUIC protocol used in HTTP/3 seems to be the workaround for that. The following recent comment and the first video linked actually had some great related info that I was recently reviewing on that topic: https://news.ycombinator.com/item?id=32572825

I don't know if the MySQL protocol itself though be able to utilize the TCP-based TLS 0-RTT functionality or not however so connecting via a regular client may still end up with a lot of the back and forth handshaking.

The newer serverless driver for JavaScript has some opportunities to take advantage of QUIC within HTTP/3 in the future as Matt mentioned over here recently: https://news.ycombinator.com/item?id=32513043

So that will be interesting to continue seeing how it evolves/improves over time.

orware

4년 전discuss

You will generally run into different types of latencies in this case since you have the general connection latency when those are getting established plus the regular physical/network latency between where the database is located and your own servers.

For connections, since a TLS handshake is required, the impact of physical distances can have a greater impact on the connection time. The following article: https://sking7.github.io/articles/44961356.html actually provides a good 3.5x-4x figure which correlates with some connection tests I've completed.

In other words, if an initial TCP packet takes ~100ms to get from the database to your server, then establishing a TLS connection to the database will probably be around 400ms.

Once the connection is established, running queries over an open connection is generally going to be quicker, at least for simpler queries. More complex queries will still take whatever time they need to process on the database end before they can start sending results back so results will generally vary there.

But going back to that 100ms example...if the amount of data being returned from a very simple query is minimal than the response time here would be very close to that 100ms figure over an already open connection and likely would go up from there depending on the complexity of the query and amount of data needing to be returned.

Since the connection hostnames are publicly accessible and TLS is always required for connections you can easily test from your own provider's location. So long as the general physical location isn't too far away from a supported region, the latency overall shouldn't be unusable.

I may have mangled some terminology/analogies above but hopefully that helps provide a bit of a ballpark for you. If you have specific to/from regions in mind I might be able to try and collect some specific numbers for you!

orware

4년 전discuss

That wikipedia article certainly brings up some need for pause/concern.

Recently, I heard about this relatively new company named Quaise: https://newatlas.com/energy/quaise-deep-geothermal-millimete...

And I thought that the technology seemed quite interesting, so I hope the development of it can pan out since it seemed like it would enable greater usage of geothermal energy around the world, which seems to be mostly untapped at this time, and also seemed like it might be able to address the seismic issue/concern, although I'm not an expert on the topic so I could be wrong there.

Geothermal is mainly a topic of interest for myself since there are a few local plants in my own region but I hadn't really realized that it is an underutilized power generation option until I read the article on Quaise recently above.

orware

4년 전discuss

I'm not an expert on this topic, but one author in particular that I came across as a younger man was James Allen and his writings: https://en.wikipedia.org/wiki/James_Allen_(author)

And in particular there is a 3-volume series that was available for a few years that put together a lot of the work he had done during his lifetime, even though his most popular seemed to be "As a Man Thinketh".

The Wisdom of James Allen I, II, and III: https://www.amazon.com/Wisdom-James-Allen-Including-Prosperi... https://www.amazon.com/Wisdom-James-Allen-Difficulties-Trium... https://www.amazon.com/Wisdom-James-Allen-III-Heavenlylife/d...

It's been a number of years since I last read them, and unfortunately the publisher above went out of business so you can generally only find the titles above used, but I did enjoy the philosophy/thinking shared in the writing, even if their titles may indicate a somewhat religious slant, overall I'd say the writings focus more on leading a good life.

This is a good reminder I should read them again to refresh my memory on all that is discussed within the pages however, since it has been probably over 15+ years since I first read them thoroughly.

orware

4년 전discuss

Quick question…did your account have a history and past positive reviews from any purchases you had made in the past?

I have a long-standing account and rarely buy on eBay nowadays (I’m trying to recall if I’ve ever sold anything…if I have it was maybe only one item but I don’t even recall if it sold or not).

Recently, I was looking into buying a used gaming PC via eBay to save a few bucks and I ended up completing a “Buy it now” purchase quickly without looking more into the seller (or their location). The location wasn’t a big deal (Paris, France) but that mainly meant the shipping would take longer. What ended up being more concerning was the 0 rating for the seller, which immediately make think “oh crap”. I reached out to the seller just to see if I could get a response with no quick reply, but I sent one short follow up the next day when I didn’t receive a response and shared my concern and waited another day before reaching out to eBay about the concern I had about the seller (especially because by this point the original listing was gone and then it even seemed like the seller’s account too). I used the live chat option and the person there was very helpful and got the process started and mentioned to reach back on Friday (about 3 days later), but later that same day my refund was issued and the case closed which I was grateful for.

But it did make me wonder of what might be an apparent difficulty for newer accounts to sell successfully on the platform? (Kind of like stories I’ve heard about liquor licenses being grandfathered in for certain locations in cities, whereas it may be more work for a new location to apply for one…maybe newer sellers can easily be flagged? The inability to dispute the situation when you are obviously willing/able to communicate with the eBay staff however is the sad part in your story since legitimate individuals should always have recourse to be heard in these large tech platforms).

orware

4년 전discuss

Another note…based on what I know, most colleges are suffering from low enrollment too, and even though funding isn’t solely based on the the student count on census date anymore (which is usually about 1-2 weeks into a semester…and faculty is supposed to drop students that don’t come on the first day of class typically)…since now the formula is more complex with “success” related factors added (numbers of degrees / certificates awarded plus some other ones I don’t directly recall right now.

This leads to number of students still being a pretty big factor in the funding received for the year. From Administrators I would say if any are worth their salt, ignoring any sort of fraud would be a no-no so I’m hoping that’s not a common situation being observed. On the other hand, losing a substantial percentage of your current budget due to a loss of students can be pretty tragic for the staff working on the campus. Budget reserves can usually be dipped into for a period of time, but what most folks don’t know or realize is that compared to private businesses where the cost of employees may be only a fraction of what the business brings in profit, most CCCs are likely spending 80-90% of their budgets on salaries and benefits for their staff (in some cases the % may be more, in some cases it may be less). This makes it extremely difficult most of the time to weather a big loss in students because if budget reserves get expended, and student numbers don’t improve, that’ll mean some sort of layoff process…which also provides those employees with reinstatement rights too for a considerable period of time afterward).

orware

4년 전discuss

XAP is the old system, but I don’t think it’s in use anymore (back then it was just “CCCApply”).

The current system, now called “OpenCCCApply”, was mostly developed by a team from Unicon (https://www.unicon.net/), along with folks within the CCCTechCenter team of my memory /understanding is correct (I’m not sure if it’s managed jointly though, or if only a CCCTechCenter team mostly manages it now though since I never heard those types of details).

A redesign/modernization effort however sounds…expensive.

orware

4년 전discuss

It’s interesting to see this thread crop up here, as I’ve only recently left CCC system for a job in tech after being in IT within the system for about 14 years.

This particular issue over the last year or so has gotten worse, with more eyeballs on it, once actual money became involved (before it was still an issue but on a smaller scale mostly for the free edu emails we tend to issue, along with other freebies that can help enable, such as free credits for Azure or other services).

Even in the previous cases, I was annoyed/upset because in my mind the first line of defense the colleges have is preventing these fake users from being able to submit an application successfully in the first place, since the OpenCCCApply application (which I believe is used by all ~115 CCCs) was allowing the submissions in the first place…and since we mostly bring that application data into the individual colleges, not many triggered a “hold” on our end.

Yes, CCCTechCenter (which helps manage the team which maintains the OpenCCCApply system) have done a few things over this past year that are mentioned in the article already (but based on the article I can’t tell 100% if it is really indicating the issue is still rampant in the more recent semesters…one of the changes was adding usage of an IP reputation checker in for example, but there are likely ways around that too for these folks who actually don’t seem to actually use bots…maybe they use actual people instead based on what I’ve seen, such as the YouTube video shared in the article).

What I found really annoying by it all is that while the problem originated from the systems being provided to the colleges from the state level (OpenCCCApply mainly), the individual colleges are now on the hook to gather a bunch of mostly useless data, and go on silly adventures such as investigating IP address info within our other systems (like Canvas) to help find or report the fraudulent activity.

I think I saw FAFSA mentioned a few times but I don’t think there is a ton of fraud coming from the FAFSA application too directly…but in this past year many of the colleges have been putting COVID relief funds they’ve received (to help get students/staff back on campus) and using those to pay for fees or provide an extra amount for books, etc. which isn’t something that will continue forever (in fact, I think for this summer this will already have ended, or it will be the last semester where it will be offered).

In most cases, once the incentive is taken away, or the bar to get it is made higher, these folks creating the fraudulent accounts will generally move on (or target schools that don’t implement some of the 2nd layer fixes at the college level…unfortunately while the CCCTechCenter tries its best, it doesn’t typically fully acknowledge its role in creating some of these situations, and I almost lol’ed when I saw towards the end of the article I saw they are looking to get more funding to “modernize” it, yet again, considering a lot of effort / time / money already has gone into creating the current OpenCCCApply system not that long ago from the previous system, which was pretty bad in comparison).

Overall though this particular situation is at the same both more complex and simpler than folks may think, once you have some more details (more complex because there is a lot about what’s going on in the CCCs the HN community isn’t aware of, along with super strict regulations that have to be followed within the individual Financial Aid departments at each school, otherwise they win not be able to provide federal aid monies to students if they weren’t doing so, making that avenue for fraud a lot more less likely than the scenario I shared above on how the COVID relief monies have been being used instead to provide an incentive to get students back in the classroom…along with the solution being simpler since we already have a central application process that should be the system that keeps these applications from ever reaching the individual colleges, but it fails in that regard…that along with removing the financial incentive currently present, should reduce the fraud levels considerably…although there are likely a few more complexities even I am unaware of…I would just appreciate it if the search for fraud wouldn’t get pushed onto the individual colleges in these situations where a system wide protection should have prevented the situations in the first place, mainly because it causes a ton on unneeded busy work at the colleges keeping IT System Analysts and other technical folks from focusing on other, probably more important, internal projects).

Excuse any typos…I wrote this small novel on my phone.

orware

4년 전discuss

It's still a bit early for the company but if you like RDS, PlanetScale might be worth a look too: https://planetscale.com/

orware

4년 전discuss

Check out my comment above here in this same thread which basically confirms exactly what you just shared here...this is the same tactic I was observing being used with fake student accounts sharing with outside external Gmail addresses which then appeared to take over the Shared Drives being created in GSuite, allowing for TBs and TBs to be added in to the drives by outside users. It seems like a big problem to me...but at the moment I don't think Google is providing good enough tools for Admins to see/eliminate these situations very quickly/easily.

orware

4년 전discuss

Just coming out of a higher education position I held for about 10 years...we were really only using GSuite for Education on the periphery of everything else we were doing (which was all Microsoft-based). So we were really only setting up the needed GSuite integrations to give some access for students/faculty/staff to use the GSuite apps if they wished, so the "unlimited" storage side of things was an extra bonus.

While our actual students/staff have not abused the unlimited-ness at all, there's been a big trend in recent years for fake spam/bot type accounts to submit online applications (particularly for the inexpensive community college level here in California) which would allow for those fake people to get issued accounts within the college systems (typically giving them a student email account).

In essence though, one bad account could in turn create multiple shared drives it seems, and then from there grant access to other external Gmail accounts and swap the ownership to them, while still leaving the unlimited storage capabilities in place (at least that's been my perception of what I had seen in our instance). Before I left recently, I had seen pages upon pages of shared drives in our GSuite Admin area of Shared Drives that were terabytes large (one as high as 100 TB or some ridiculously high number like that)...with very little mitigation options being provided by Google to help avoid that from occurring in the first place (aside from taking the extreme measure of locking things down after the fact...but cleaning up that existing mess seemed to not have a solution...supposedly there were going to be some new tools released early this year to help out, but not sure if those are available yet).

Suffice to say...some bad apples are making it hard for all of the good people to take advantage of what was otherwise a wonderful offering for educational institutions (having administered many systems over the years...it's always nicer to not have to worry about license limits and other baloney like that since it simplifies things for what is usually an already overloaded IT staff).

I could clearly see all of the bad accounts abusing our instance...if only there was an easy way to filter all accounts > 2 TB and then delete them I could have reduced our storage usage down considerably for Google.

That multiplied by hundreds of institutions where this may have occurred during the pandemic and I can see how storage usage could have went up dramatically on Google's end...but I think that Admins in those institutions would be more than happy to clean things up if they had the tools/ability to do so.

orware

5년 전discuss

I'm not sure I understand the criticism in the top comment in this thread about Caddy's performance.

I don't think I ever really decided to use Caddy solely based on performance itself (although already fairly familiar with Go's built-in libraries for creating HTTP servers gave me confidence that it should be fairly quickly...I wasn't overly concerned about how it directly compared to Nginx/Apache/IIS/etc.).

For me, the biggest selling point was the Caddyfile format, which felt a lot more human readable than the .htaccess or Nginx configuration options I had seen over the years (I know with Caddy v2 there's the possibly now more preferred JSON option, but I'm still a sucker for the original Caddyfile format myself :-).

There's still a lot I don't know about for all of the web server options out there in general so I don't claim to be an expert in any of them, but Caddy was the first one I felt comfortable using new situations to simplify things in my environment.

For example, Caddy's reverse proxy functionality (and the way it is setup within the Caddyfile) is what finally made that particularly capability "click" for myself over 5 years ago when I first came across Caddy (even though I had seen info how Nginx + Apache were used together in the years prior, with Nginx providing reverse proxy access to Apache in those hybrid setups, it wasn't something that had "clicked" for me being aware of those situations). Since my environment is fairly simple (no real super high demand situations requiring lots of load balancing) we mainly use Caddy to provide automatic SSL capabilities and act as the publicly accessible endpoint in our environment, and use the reverse proxy functionality to tie Caddy into the various other internal servers that need that SSL termination.

I've also been able to take advantage of the built-in static file server for an internal website need and it's actually something I really think is a nice/neat feature to have available (especially due to its ability for simple server side includes, allowing for relatively easy simple templating).

Separate from that, I've also experienced first hand the personal responses that Matt Holt has provided to myself and to many others in the Caddy Community, and I'm definitely appreciative of the work he's put into Caddy (along with others of course) and I hope some of negativity in this thread is able to be easily shrugged off by Matt, because I could imagine it can be exhausting to deal with when he genuinely does care and puts a lot of thought/effort into the responses he provides.

orware

5년 전discuss

Just an FYI, it looks like your 2-year GoDaddy Wildcard SSL Certificate expires today (Chrome sent me to the "Privacy Error" page when trying to visit your site a moment ago so I took a quick look at the certificate).

orware

5년 전discuss

I would love to just focus on dozens of problems / interesting areas to myself, but those of us without healthy bank accounts don't really have that luxury.

I really wish there was a sponsorship model of some sort where folks that just love programming for the sake of it could be sponsored by those wealthy enough not to worry about money anymore.

While there are still a lot of interesting problem areas within a typical workplace, sometimes the interests one has within programming don't necessarily align with the workplace's needs, making it so that to work on the interesting stuff in your own time, which gets harder as "life" happens and bills become plentiful.

I too though read about those famous folks in science and noticed how they had either financial sponsors, or were themselves already wealthy, and that allowed them to pursue their own learning paths in earnest, where I would say that isn't an option for the vast majority of us today (it also circles back to general opportunity...there are so many smart and intelligent people out there that may never get the chance to reach their full potential due to the lack of opportunity and instead have to focus on more basic needs, etc.).

orware

5년 전discuss

Although my unique circumstances are different, I'd like to share that you are not alone in feeling trapped in one's job and city since I too have felt the exact same way in recent years.

I unfortunately can't offer very many solutions since I am in basically the same "trapped" situation currently (but towards the end here I can hopefully share a few things that keep me going).

For myself, the position I currently have in my hometown is the best I can hope for in this area, but the work/team environment have steadily become worse, particularly the last few years...being able to work from home last year helped out tremendously since it was something I had been hoping to have for some time, but the main kicker was simply being able to be around my kids more and have those smaller interactions with them on a more frequent basis and not deal with the office drama constantly.

Age-wise, we're the exact same, but education-wise you're further along than I'll likely ever get (I have some work towards a Masters in CS, but I'm poor at math, so with a completed PhD in CS in hand for yourself, I feel that opens up so many more doors than I could hope to receive). Housing-wise, I fortunately live in a somewhat rural area so housing prices aren't nearly as high as they are in other parts of California so we were able to take the plunge to purchase a home about two years ago (at times I feel that renting was definitely easier, but we were outgrowing our old place so I echo the sentiment of having one's own place to make "home" as being a positive).

I think the big thing I'm reading in your post above though is potentially that proximity to family and a city where you actually do feel like you're home and I can also echo that sentiment too. I spent a year in the Bay Area for school, but always felt that it was different than my home area, and later on during a short stint at a tech company in the Bay as well, those sorts of feelings returned too (it's difficult for me to say though whether I might have eventually felt "at home" if I had given myself more time but I'm not sure if that would have occurred or not).

In the last month my workplace has had myself return to the office and it created an immediate sense of sadness since I would no longer have the comfortable environment I had built up at home, nor would I get to see my children as often I had grown accustomed to. That I feel is the great drawback of so many companies pushing to bring their staff back onsite rather than adopting proper remote work options and allowing for it moving forward.

On top of that, I took a short (less than a week) vacation with the family, and had similar feelings as yours in my first day back in the office afterward since I just resented the thought of coming back into the office again so much after enjoying a few days away.

Also like you however, I've even shied away from wanting to apply for Software Engineering roles at other companies because I have the same concern regarding whether I would prefer the new work more than my existing work...or would it in fact be worse? That's the big concern (aside from the larger questions of whether remote work would be allowed and if I would be forced to make larger changes such as completely moving my family away to another city, etc. which also doesn't seem as appealing to have to put them/myself through either if it can be avoided).

So I wonder, what's my alternative to this line of work I've been in over the past decade+? And it's a really sobering thought since there would basically be nothing (that I can think of) that would come close in terms of salary so I feel forced to continue on in it (even though I do enjoy software development overall, I've also worked mostly independently since I'm basically a team of one, so I'm never sure if working in a larger software oriented organization might take away some of the enjoyment I've had working independently on software projects due to the extra rigor that might be needed on larger teams...overall though I'd still hope that the extra comradery would be a net plus on that scale).

Since you mentioned that your partner earns several multiples of your current salary I'm not sure what kind of accelerated timescale that might provide you as a couple to a potentially early retirement elsewhere (potentially closer to your hometown), but that might be consideration (although it doesn't help out with the here and now feelings which can't be worked around if you're still needing to go to work and experience those feelings daily).

For myself, it is difficult too...in years past what would sometimes help is me making attempts to get hired at tech companies (providing some small hope for that short period of time before being eventually rejected...which is also a not so great feeling so that's one reason why I've shied away from doing much of that lately), but at the moment I'm actually exploring Y Combinator's Cofounder matching area to see if that might provide me one path away from my current role I feel so stuck in and maybe take me into a new role (e.g. CTO) where I can provide more impact in a smaller company and get to stretch more of my brain and what I feel I can bring to an organization. I'm not sure if this experiment will pan out or not, but I'm hopeful for now :-).

Feel free to reach out via my email in my profile if you feel like sharing any more of the "struggle" we both seem to have (it's good to know I'm not alone in feeling that way either :-)!

orware

5년 전discuss

I had been trying to get into Tailwind CSS in recent projects but noticed some of the Bootstrap functionality I tend to like was missing out of the box so with Bootstrap 5 coming out last week I switched gears to start using that for a new project.

The project itself is an update to an older one that was originally built with Bootstrap 2.

For the old one, when I go to print the somewhat detailed tables I'm using show up fine, but in the new Bootstrap 5 version, one of the tables sticks outside of it's container when going to print and for the life of me I've been struggling to figure out a solution that works consistently for the print side of things (which I feel tends to be overlooked quite a bit in regards to CSS). What's even odder, is that when I implement a change that helps with the second table (making it appear contained within the page dimensions) the first table above starts exhibiting the issue of it being outside of the container. Also, in this case, if I were to leave the default styling of the "table-responsive" div containing the table, you wouldn't be able to see the rest of the columns at all when printing (since the scrollbar would show up) so everything I'm describing above is with an override to have "overflow-x: visible" when printing. Some other suggestions online included adding some overrides that use "page-break-inside: avoid;" for tables/tr/td/div elements and "display:block !important;" for the body and div elements but I'm not completely happy with those results, but yup that's been a bit frustrating trying to sort out since I'm not even sure why it's behaving the way it is at the moment.

I haven't kept up fully with all of the Sass/SCSS advancements in recent years but have a semi-decent CSS foundation to build off of (from my understanding those advancements primarily help with some additional programmatic elements that then get pre-processed back into CSS? Along with some syntactic sugar to simplify some of the repetitiveness that CSS can require).

The other big annoyance I was dealing with was related to what appears to be a long-standing WebKit issue related to scrollbars on mobile devices? (for the theme I'm using, I'd scroll down on an iOS device and wouldn't be able to see the very last item in the mobile menu). I found a reference somewhere (probably Stack Overflow of course) that mentioned using "@supports (-webkit-touch-callout: none)" to try and target the mobile browsers specifically and I reduced the height by an extra 65px or so which seemed to be enough to help with that particular issue (the theme utilizes "perfect scrollbar" it seems, but not entirely sure about all of the styles involved). (Before settling on this solution I had already tried out the solution described over here: https://allthingssmitty.com/2020/05/11/css-fix-for-100vh-in-...).

One last item I was dealing with this week seems to have to do with SVG rendering on the desktop vs. on mobile (for a logo). We sort of have a designer on staff but he deals more with print than with web stuff so I asked for a variation of our logo with the name of the application integrated into it and provided back to me as an SVG (since the PNG I had created myself looked slightly fuzzy when it was downscaled to fit the space I was using it in on the page, so I felt an SVG version would look crisper). After swapping things to the SVG version provided to me, it looks pretty good on mobile, but on the desktop (both in Chrome and Firefox), things just don't look as good. Some resources online mention it could be related to the lower DPI of a desktop monitor vs. the retina type DPI of a mobile device, but I don't know...I also saw some mention of a "pixel hinting" approach that might be able to be utilized by the designer in Adobe Illustrator perhaps that might help as well so that's something I've asked him to give a try for me so we can see the results, but I was kind of disappointed about the way the SVG looked as well (and now that'll require a bit of extra time to research/learn/experiment about, rather than focusing more on the application I'm trying to actually build...but at the same time it's semi worth it at the moment since I'm hoping the foundation I'm building with this new web app will be utilized in a couple of other updates that I'd like to make over the coming year, but we'll see how it goes ;-).

I'll postscript this message that I'm not a frontend dev, but since I'm basically a team of one with an eye for certain details/issues, I try and get these sorts of problems sorted out at the beginning of a project so they don't cause issues, but also as a team of one, I really wish that in addition to the backend frameworks that are available (Laravel in my case is the one I'm currently ramping up on, as it's the more or less leader on the PHP side), it would be nice to have a nice frontend option with "batteries included" for a developer to really utilize and not have to worry too much about (Bootstrap has probably been the closest in that regard...I do like a lot of what Tailwind CSS is doing though, and of course it's heavily promoted in the Laravel community, but I just found that it would require too much work for me and too much time...and my workplace isn't exactly software development focused so I have to pick my battles in that regard vs. spending a lot of time getting something basic going vs. getting something shipped for users to start using).

Hopefully one day we'll be completely away from needing to do any sort of CSS hacks (and things have come an extremely long way in the past few years), but at the moment I wouldn't say things are completely hack free yet (at least for us non-100% frontend folks...you all might have some extra magic you can cook up on the fly to solve any problems encountered maybe using what's already available :-).

orware

5년 전discuss

As an aside (or perhaps related note), over this past weekend I was doing my first real-world usage of SQLite, since I typically rely on conventional databases for my day-to-day work.

I've read the article shared here in this thread in the past and read about the usefulness of SQLite generally, particularly for read-heavy situations, so I thought it would be a nice experiment to pull in the "Have I Been Pwned Password" List and import it into the SQLite and see how fast it could be queried.

At first I didn't find too many resources out there of the same thing having been done, but after searching some more throughout the weekend I did find out some nice nuggets of information that helped (in particular, I learned that adding the "WITHOUT ROWID" option at the end of a SQLite CREATE TABLE statement will help to reduce the overall database size considerably).

The desire to put things into SQLite for the "Have I Been Pwned Password" actually stemmed from a misinterpretation of the API page since it seemed at first that the API was not free (although I had thought it was), since one of the first things I read was the need for a $3.50/mo charge. Later on, I then figured out that the password checking API is still free, it's just the other pieces of the API that require the API Key / monthly charges (doh!).

If you run a query against Troy's API, it's pretty damn fast all things considered, so I'll likely revert to using his API directly, but I was thinking of uploading the SQLite versions to some cloud hosting somewhere and potentially share it with others if anybody is interested.

On to some quick stats collected over the weekend (using an import process powered by some PowerShell):

  50 million took ~6 hours 22 minutes to complete (with ROWID):
  Start Time: Sunday, February 14, 2021 9:12:29 AM
  End Time: Sunday, February 14, 2021 3:34:46 PM

  50 million took ~5 hours 46 minutes to complete (without ROWID):
  Start Time: Sunday, February 14, 2021 7:51:38 PM
  End Time: Monday, February 15, 2021 2:05:30 AM

File size comparison:

  5.28 GB (50 million rows, with ROWID)
  2.61 GB (50 million rows, without ROWID)

Search Time comparison:

Example performance of Select-String (line-by-line search):

  TotalMilliseconds : 216245.7246

Example performance of the SQL query:

  TotalMilliseconds : 20.8543

I then ended up finding this implementation of a Binary Search option (rather than the naive approach Select-String takes above) that Chris Dent put together here: https://www.indented.co.uk/powershell-file-based-binary-sear...

Example performance of this approach was more in line with the SQLite database:

  TotalMilliseconds : 40.3749

(For these runs comparing against the Binary Search option, the SQLite query ran in):

  TotalMilliseconds : 4.9491

All in all, now that I have discovered the Binary Search option that one would be workable without needing to do anything at all (except to make sure to download the "Ordered By Hash" version of the hashed password list) and use it directly, although I believe once the import has completed of the full password file into SQLite (using the WITHOUT ROWID option) the file size for the database should be fairly close to the actual txt file size (the text file size is 25.1 GB currently with close to 630 million lines in it...based on the smaller batch conversions to SQLite, I think the full import should be in the low 30 GB range).

Tagging along with the top comment in this thread however, I do see what they mean about insert performance since I did try a few experiments there as well, but went ahead and kept things with the 10k batch inserts at a time (I think with individual inserts I was calculating things to potentially take 216 days or something ridiculuous, and switching to batch inserts made it go down to about 3 days, but based on some comments I read elsewhere over the weekend, even that seemed like it might be kind of high...at the moment I'm trying a full import outside of my PowerShell script using DBeaver's functionality for doing so, so I'll see if it completes faster...seems like it might since based on the current database's size it seems like it's more than 50% complete and it's probably only been about 12 hours).

orware

6년 전discuss

I arguably have a pretty good job currently in the public sector which has a fair amount of autonomy and prestige that have been earned within my current organization over ~13 years.

Last spring after COVID hit I was reached out to by a Facebook recruiter with the conversation morphing after a few months (and two other individuals on their end) until having a really exciting conversation with a recruiter in early August that was just refreshing since it actually seemed like Facebook's hiring process wasn't that bad and that they might actually value me and my existing experience.

Part of my challenge is that my current role is not completely coding focused so I really don't have that "true" software engineering background and find it hard to figure out where I'd fit in a typical tech company's role structure. Instead, I have experience coding my own projects for my organization and taking them from concept to production in weeks and maintenance over timespans of years, along with management and tech leadership experience within my own organization.

Since coding isn't a daily focus all the time, my programming skills tend to need to be dusted off each time I take on a new project, and I know I'm not the best algorithmic expert, etc. but I'm reasonably intelligent (in my own humble opinion) and able to learn from others when I have that opportunity available (learning about extreme scaling techniques and certain advanced topics that might only be encountered within a tech-focused organization is unfortunately something I can't easily do within my current organization, and team-wise I'm the primary person interested in these sorts of topics so don't have others to really discuss the ideas with, which leads to that interest in potentially moving to a tech-focused company). I feel like my existing experience and ability to work with others and learn quickly would have some value (especially when you consider that new graduates don't necessarily have a lot of these experiences under their belt, but they are still considered for hiring...their algorithm knowledge might just be a bit fresher/recent).

Switching back to the interview...I spent a bit of time studying for the initial coding interview and felt like I had done decently well, but still received the "thanks but no thanks" email a few weeks later.

After that rejection (along with others over the last several years from companies I saw on the monthly "Who's Hiring" threads that intrigued me) I'm kind of done with trying to apply for tech jobs at the moment.

I'd love to earn more and work remotely from my current location (now that the pandemic has made this a little more possible) and be able to contribute my working days to contributing to the success of a tech-focused company, but it's not easy to be accepted by a tech company when you don't have the right background or exact skills upfront, so at least for the time being I need to be happy with my current work situation (even if it isn't always the most glamorous, tech-wise, and doesn't pay the same as a large tech company might).

This is just one personal anecdote of course, but I do wish there was a different hiring process for folks wanting to break into a tech company with existing experience that would be valuable, albeit non-traditional to typical hires.

Submissions

The Fight over California's Ancient Water

ACM Siggraph 2020 Doctoral Dissertation Award Goes to MIT CSAIL’s Tzu-Mao Li

Ask HN: Have “Who wants to be hired?” threads worked for you?

Ask HN: Acceptability of Public Sector Experience for Job Hiring

Show HN: PHP Compressed Strings Library

Ask HN: For Any HN Googlers That Work on Chromecast

Ask HN: Examples of Good Software Development Contracts

Show HN: Sharing Who's Hiring Data Project with Community

Ask HN: Would any startups consider setting up shop in Imperial County, CA?

Ask HN: Hiring an Employee in a Low Income Sole Proprietorship

Commercial Joomla Extension Business Review (2010, 2011, and 2012)

comments