gwittel · HackerTrans

gwittel

429 karmajoined 7 lat temu

Computer security Anti-spam / malicious content filtering Backends Scaling

glwittel [at] gmail [dot] com

comments

18 dni temudiscuss

Chronicle has a lot of great resources. I’ve been out of Java for a few years but when I had to write high perf code, the libraries and blog insights were invaluable.

A lot of the advice is good in general - keeping things simple, generating fewer objects, etc. Profiling with Yourkit or JMH to find and improve slow spots.

This let us build high performance software that had a vast difference in say p90 input size and p99. It involved rewriting third party libraries to let us 3x speed and substantially reduce GC. I think in the end we were around a p99 of 7ms and p90 well under that (the data spread was kbs to megabytes) across hundreds of millions of inputs per day.

w zeszłym miesiącudiscuss

Pretty much this. We see CF hosted/protected content all the time. CF does nothing for days or weeks. By then the campaign is gone and on a new account etc. Tycoon and Kratos are two phish kits that heavily use CF.

CF does have small anti-abuse teams, but it’s just not a business priority for the company to do better. We’ve tried many times to engage at a corporate level and the bottom line is they don’t care - they don’t want to police content as often stated by the executives.

w zeszłym miesiącudiscuss

Given how little they do now to stop malicious content hosted behind/by Cloudflare, the bare minimum if anything.

3 miesiące temudiscuss

> "GitHub only gets better if people who give a shit stick around to make it better"

At a basic level I appreciate this sentiment. However, the common dysfunction I see in large corporation is its not the lack of people who give a shit. Its lacking a sufficient number of people in positions of power that give a shit -- such that they can actually make change happen.

All too often competing pressures (features, profit, delivery speed, politics) take precedence; not leaving time for things that would really move the needle. In essence, too many leaders are happy to ship garbage; they don't care (or don't know).

If Github were to put out a statement saying "service quality is our priority", it is fairly meaningless. If they added "here's how we'll get there", maybe it helps some. Moreso -- "from now on executive compensation is tied to these SLOs", then maybe something would actually happen.

3 miesiące temudiscuss

Yes. In the past I helped sort out tooling like this for competitive analysts. There are a few ways this is done:

1) Check the businesses’ MX record. Often this points to a third party provider like Microsoft or Google. 2) Connect to the mail server identified in the MX record. Sometimes these have banners that identify the vendor (vs something generic like sendmail) 3) Email headers from messages sent to users in the company (or sometimes a bounce). Often these have headers from one or more providers. You’ll have to sort out the path to understand which bits were added by the sender/recipient path though.

These days often companies have multiple providers (security) so they might have one at the edge (mx) and more internal hops. You can usually see these in the headers.

7 miesięcy temudiscuss

Yes. I remember listening to it on the radio. The DJ used the handle Hard Hat Mack. It was pretty awesome to hear SID music over the radio.

I found this archive that has some of the shows recorded, and set playlists (I'm giving two links as the site is using frames so the top level page requires you navigate the menus to get to these):

  - Recordings: https://www.transbyte.org/SID/KDVS.html
  - Playlists: https://www.transbyte.org/SID/6581.html

The set playlists (using HVSC) works. For actual recordings they're 404 from this site -- arnold.c64.org is gone. But there are a few archives of the arnold.c64.org site! This should help re-construct the original links from the above page:

  - https://www.mmnt.net/db/0/0/arnold.c64.org/pub/sidmusic/lala/ra
  - https://archive.org/download/arnold.c64.org  (download the whole thing and dig into pub/sidmusic/lala/ra)

Due to the era, most of the files are in RealAudio format; with a few MP3s as well. Wonder if this could all be re-posted somewhere in modern formats to make it more accessible.

Its possible the authors are still around and have more copies; doubtful KDVS has archives, maybe tapes buried in the library.

Anyway, hope this helps! Its a cool piece of history and brings back a few memories.

8 miesięcy temudiscuss

Tools like Playwright and Puppeteer are abstractions on top of CDP. The other use case is when these frameworks don’t expose or don’t use a CDP command you need (often they hide some parameters for cross browser compatibility).

Webdriver BiDi is a future cross browser replacement:

https://www.w3.org/TR/webdriver-bidi/

9 miesięcy temudiscuss

I’ve had a similar thing happen to me recently. 500$ tariff on $130 of stuff. The tariff should have been like $20. UPS has been completely non responsive and still won’t show me the customs forms. Total scam.

w zeszłym rokudiscuss

I work a product that involves a security crawler (phish, malware detection, etc). It’s just a new arms race. Crawlers will adapt.

Cloudflare is already heavily abused by threat actors to host, and gate their malicious content. This means our crawler has to handle anti-bot and CAPTCHAs. It’s a pain. Cloudflare is no help.

They have a “verified bot” program but it’s a joke for security. You must register a unique, identifiable user agent, and come from a set of self declared IPs. Cloudflare users can check a box to filter these bots out. And now you're easily fingerprintable so the bad guys can just filter you even without Cloudflare’s help.

So now we have a choice. Operate above board and miss security threats. Or operate outside the rules (as opaquely defined by Cloudflare), and do right by our customers.

All of this on CFs side is to solve a real problem. Unfortunately by not working with the industry in a productive manner, Cloudflare is just creating new problems for everyone else.

w zeszłym rokudiscuss

Oh fun. I can’t wait. Now phishing sites will be protected with Turnstile and this garbage.

2 lata temudiscuss

Interesting. Looks really neat! How do you deal with anti bot stuff like Fingerprintjs, Cloudflare turnstile, etc? Maybe you’re new enough to not get flagged but I find this (and CDP) a challenge at times with these anti-bot systems.

2 lata temudiscuss

Google search results are full of garbage pages populated with LLM generated content (the pages exist solely to serve ads and capture search results).

Search spam is not new, but the use of LLMs simplifies the ability to make pages that look like legit content (increasing the likelihood they’ll show up in search results).

2 lata temudiscuss

Neat optimization! Would it have been feasible to spend a bit of cpu/memory and tag headers as internal upon the request construction? That way filtering on output is trivial.

2 lata temudiscuss

They could police their content. Or if they don’t want to, they could meaningfully partner with the security industry - create a “security bots” program, respond to takedown requests in days not months, etc.

2 lata temudiscuss

You can. Sort of. The good bots list is basically driven by a fixed user agent. And customers can set their preference to not allow “good bots”.

Not so good for security work.

It’s similar to their abuse reporting. They give your info to the site owner. Gee thanks, that’s just what I want to do.

2 lata temudiscuss

I’m really mixed on this. Anti bot stuff is increasingly a pain point for security research. Working in this space, I have to work against these systems.

Threat actors use Cloudflare and other services to gate their payloads. That’s a problem for our customers who are trying to find/detect things like brand impersonation and credential phish. Cloudflare has been completely unhelpful. They just don’t care.

3 lata temudiscuss

Cool library. This sounds like SWAR (simd within a register). I’ve seen these techniques give a nice speed up especially when SIMD isn’t available or a pain (eg in Java pre-Panama).

One random sample:

https://lemire.me/blog/2022/01/21/swar-explained-parsing-eig...

3 lata temudiscuss

I was looking at [1] recently to understand omicron variant positivity length and they cite a few other papers. The article [1] is publicly available. I haven’t checked if all of the others are.

* Routsias JG , Mavrouli M , Tsoplou P , Dioikitopoulou K , Tsakris A . Diagnostic performance of rapid antigen tests (RATs) for SARS-CoV-2 and their efficacy in monitoring the infectiousness of COVID-19 patients. Sci Rep. 2021;11(1):22863. doi:10.1038/s41598-021-02197-z

* Currie DW , Shah MM , Salvatore PP , et al; CDC COVID-19 Response Epidemiology Field Studies Team. Relationship of SARS-CoV-2 antigen and reverse transcription PCR positivity for viral cultures. Emerg Infect Dis. 2022;28(3):717-720. doi:10.3201/eid2803.211747

* Korenkov M , Poopalasingam N , Madler M , et al. Evaluation of a rapid antigen test to detect SARS-CoV-2 infection and identify potentially infectious individuals. J Clin Microbiol. 2021;59(9):e0089621. doi:10.1128/JCM.00896-21

* Killingley B , Mann A , Kalinova M , et al. Safety, tolerability and viral kinetics during SARS-CoV-2 human challenge in young adults. Nat Med. 2022;28:1031-1041. doi:10.1038/s41591-022-01780-9

[1] COVID-19 Symptoms and Duration of Rapid Antigen Test Positivity at a Community Testing and Surveillance Site During Pre-Delta, Delta, and Omicron BA.1 Periods. https://jamanetwork.com/journals/jamanetworkopen/fullarticle...

3 lata temudiscuss

The Inkplate 10 is great. I haven’t gotten a lot done other than toy stuff, but so far it’s been a mostly good experience.

Another nice entry point is micropython. Some of the getting started stuff has gaps but overall nice and simple if you’re more comfortable in Python. Major libraries have ports so it mostly is an easy dive in.

4 lata temudiscuss

In a past job I’ve seen crappy crawlers from badly designed security applications do stuff like this. An an example one customer was using Trend CAS to scan all URLs in their inbound email. This causes big bursts of traffic on our systems.

The crawls came from Azure and AWS. Forged UAs, repeat hits in the same URL, etc.