twmb · HackerTrans

twmb

no profile record

Submissions

Show HN: Kcl, a Kafka command line client to do anything with Kafka

github.com

7 pointsby twmb6 ปีที่แล้ว2 comments

Show HN: Kcl, a Kafka command line client

github.com

1 pointsby twmb6 ปีที่แล้ว0 comments

comments

5 ปีที่แล้วdiscuss

Are there performance worries about passing around two pointers for anything that needs to allocate, as well as storing these pointers in a struct? AFAICT this basically means two registers are eaten, and a lot of types have effectively 16 bytes of overhead. It seems like this could quickly change the calculus on what fits within cache lines and what doesn't, which people often care about for very high performance code.

I wonder if it's possible to change the compiler to detect that, if what is being used in arguments is the global default allocator, the first argument can be stripped and all references inside the function can be replaced with the global pointer. Potentially the same concept could apply to allocators that use thread local storage. (perhaps these optimizations already exist?)

5 ปีที่แล้วdiscuss

Nice! That’s definitely a good practice :).

I’ve tried to make the kgo package easier to use, but I think some libraries suffer from over abstracting the underlying protocol, which ultimately locks them out of some higher efficiency code or some correctness. So, I’ve tried to make the library simple yet not hide all of the details. It’s seemed to have panned out successfully for at least some people. Happy to answer any questions about it, and thanks for taking a look!

5 ปีที่แล้วdiscuss

I spent quite some time writing github.com/twmb/franz-go due to wanting more from both Sarama and from Segment’s library. Pure Go as well.

I’d recommend evaluating it as an alternative, even to confluent-kafka-go due to comprehensive feature support and (afaict) even higher performance than the aforementioned libraries.

6 ปีที่แล้วdiscuss

Thanks!

It isn't yet integrated with the schema registry, but I think doing so would be a fairly straightforward task. My primary goal before this "Show HN" was 2.6.0 feature completeness.

I'll look into integrating with the schema registry within the next month. Thanks for the reminder!

edit: there's a few other minor kafkacat features that I'd like to implement as well, such as start and end times for consuming, and producing entire files. I think those two, in addition to avro schema registry, are the main features this is missing, but otherwise kcl is much more featureful.

Long term, I think it'd be great to add support for secrets in vault, but I'd like a bit more usage before considering integrating that.

7 ปีที่แล้วdiscuss

The opinion article you linked is from a right-rated source [0]. The main suggestion in it,

> That would start with the federal government imposing a national, revenue neutral carbon tax of about $200 per tonne of emissions, immediately (not $50 in 2022 under Trudeau’s plan) and then returning all the money raised back to Canadians in dividend cheques.

has pretty wide support on the left. I'm not sure how that differs from being wealth redistribution, which is what the opinion piece is against. It also doesn't describe what the current scheme is nor how it is wealth redistribution.

As far as party support for nuclear, [1]. Democrats care more about wind/solar. Neither party has majority support for emphasizing nuclear. I don't think people that care about CO2 would rule nuclear out if it were the only option.

Jets vs. thermostat type problems are tragedy of the commons. I don't appreciate private jets ferrying around political leaders, but I doubt they'll be flying commercial. That doesn't seem practical from a security nor time perspective. But, millions of people adjust their thermostats. If everybody tolerated just a liiiiittle bit less than their ideal temperature, how much would that save? Consumption is a tragedy of the commons problem.

I'd like evidence that limousine liberals are the problems w.r.t. taxes. I agree that there is hypocrisy. I disagree about how it rings hollow compared to redistributing the means, especially since "redistributing the means" doesn't even really make sense in that block of text.

Lastly, studies aren't full of blatant inaccuracies nor FUD. The world isn't ending outright, but it's getting a lot worse in a lot of ways in a lot of areas. The Syrian civil war was partially exacerbated by the worst drought ever recorded [2]. The science is settled. Exxon Mobile predicted [3] exactly what would be happening today [4] back in 1982. The conclusion of their summary is that serious adverse problems are not likely to occur until the late century, and that the time should allow for coming up with solutions. That time is now, and globally, we are doing very little, because doing what needs to be done would be economically detrimental to established interests.

[0]: https://mediabiasfactcheck.com/toronto-sun/ [1]: https://news.gallup.com/poll/182180/support-nuclear-energy.a... [2]: https://en.wikipedia.org/wiki/Syrian_Civil_War [3]: https://insideclimatenews.org/sites/default/files/documents/... [4]: https://twitter.com/UNFCCC/status/1125085040768167937

7 ปีที่แล้วdiscuss

As a recent Brave convert, I was skeptical of Brave for the same reason.

I switched to it from Firefox (after switching months ago from Chrome) due to Firefox being slow, tabs repeatedly crashing, UX disharmony, and the recent cert addon debacle.

Brave's ad proposition makes sense: the internet is inundated with ads, which is unpleasant, but free websites do need to survive. So, make the ad landscape nicer on an opt in basis.

They take privacy seriously. I'd recommend it.

7 ปีที่แล้วdiscuss

This is a decent reminder blog post about something that is obvious on the surface, but subtle when you forget to think about it and make a mistake.

I've found that thinking of a lock as something you don't want to hold goes a long way. The point of locks is to release them.

Jeff Preshing has a good series on locks and concurrency. This[0] post (potentially with an extra dozen I read in the same sitting) is the one that really changed how I thought about locks: before I read it, I was very into lock freedom at every opportunity.

0: https://preshing.com/20111118/locks-arent-slow-lock-contenti...

8 ปีที่แล้วdiscuss

I think it is fair, but I can imagine a scenario where a person reads a line and thinks "this is obviously leftist".

Personally, I think company's have been bending over backwards to prove they aren't biasing against the right.

8 ปีที่แล้วdiscuss

I caution against latching onto the title and making assumptions. The bulk of the content is analyzing where the current censorship position came from and the conclusion is for Google to be more open with their stance and also be more equal with applying it.

As for the why:

  Why the shift toward censorship?
  - User Demands
    - before, In the absence of rules, bad behaviour thrived
    - now, Appease users, maintain platform loyalty
  - Government Demands
    - before, Governments were unhappy to cede power to corporations
    - now, Respond to regulatory demands, maintain global expansion
  - Commercial Demands
    - before, It’s impossible to neutrally promote content and info
    - now, Monetize content through its organisation, increase revenues
    - before, Advertisers were wary of unintended placement and endorsement
    - now, Protect advertisers from controversial content, increase revenues

As for the conclusions:

  Don’t take sides
  Police tone instead of content
  Enforce standards and policies clearly
  Justify global positions
  Explain the technology
  Improve communications
  Take problems seriously
  Positive guidelines
  Better signposts

Most of the document is non-controversial. There are places where it can be seen as obviously left leaning, but there are also places where it acknowledges that sometimes the right has been treated worse:

  “[Richard] Spencer doesn't get to be a verified speaker; Milo gets kicked off, but I know 
  plenty of pretty abusive feminist users or left wing users,
  expressing themselves in exactly the same way
  that the right is being penalised for,
  who are permitted
  to perform certain kinds of speech. That’s going to get Twitter into

8 ปีที่แล้วdiscuss

Try the Basic HTML view linked at the bottom of https://support.google.com/mail/answer/15049.

It's is web 1.0, but it sticks, and it is fast.

8 ปีที่แล้วdiscuss

On HN, I mostly see it in replies. Then again, I don't click on all comment sections; maybe I naturally have been avoiding the articles that elicit those comments.

Generally, I expect comment sections that immediately devolve into political discussions to be flagged.

8 ปีที่แล้วdiscuss

This account looks to have been created for the sole purpose of posting this comment.

I've noticed an increasing trend of top level comments on social commentary articles immediately jumping to "liberals are the problem". Just yesterday, [0], the first reply. I do not see this happening in reverse. It is divisive and disheartening.

[0]: https://news.ycombinator.com/item?id=18099488

Further, this comment does not echo my take on the article as a whole. It's half a book review, half a person's commentary on their life in SV. The key paragraph:

> I felt like I’d ceased to become anything else,” she said. “All I did was work all the time and talk about tech.” She concluded that a job that asked her to jump from crisis to crisis, that did not admit time or perspective to consider many ideas that were outside its small world, was not the best use of her time.

The main gist of the sameness mentioned is that everything is tech.

Where is this political play coming from? Why is it a liberal problem?

8 ปีที่แล้วdiscuss

Wouldn't drawing attention to bannable offenses be Streisand effect-y?

8 ปีที่แล้วdiscuss

I think it goes to the rich through systemic imbalance in companies and personal greed of those at the top. Wages should rise.

I don't mean that people should work hard until they die; people should be able to enjoy their lives while working -- hopefully at a job they enjoy, maybe more community oriented as their needs for large salaries to support families goes away.

I don't think that is happening, I think people are nearly destitute, and I see that as the problem.

8 ปีที่แล้วdiscuss

Agreed, I think that in health cases, there can't be an expectation for people to work. In a better world, our society as a whole would have better end of life support for when people are of ill health.

But, not everybody's mind slips. I'm hopeful that we will have better preventative measures for Alzheimer's and other mind or body crippling diseases soon. I do recall seeing some good news on the Alzheimer's front in the last year.

My first article mentions what you said: past a certain point, it may be more detrimental for some to work.

8 ปีที่แล้วdiscuss

What is lost in the discussion of retirement ages being bumped is that life expectancy has grown by two decades since the retirement age was introduced in the USA in 1935 [0][1][2]. The retirement age was originally past the point where most people died.

[0]: https://www.theatlantic.com/business/archive/2014/10/how-ret...

[1]: http://www.demog.berkeley.edu/~andrew/1918/figure2.html

[2]: https://fred.stlouisfed.org/series/SPDYNLE00INUSA

I don't personally understand the expectation that average people be economically sustainable while jobless for two decades+. It makes sense to me to slowly increase the retirement age.

8 ปีที่แล้วdiscuss

It is easy to think that everybody can carve out even a small 5-10m for exercise, but that thought fails to consider people that have had or do have chronic pain or serious injuries.

It is incredibly hard to go from zero to minimal exercise, especially when even basic exercises seem to flare up old injuries. Where do you start? A poorly done crunch might throw out your back again -- do you risk it? Ten push ups might flare up your old tendonosis, should you do any? Your knees are weak from lack of exercise for years, and now 20 free weight squats makes your knees ache the next day -- is this good pain, or will the problem get worse as you do more?

Even without pain, it isn't hard to imagine people being too busy for _most_ of their day to find the motivation to exercise for a simple 10 minutes. Your day might start with a long, early commute to a tedious job. After a long commute back home, it may be hard to say "well, time to strain my body!" That is mentally tough, especially so if you have to prepare dinner, interact with kids, answer late night emails, etc.

I don't scoff at people who fail to exercise. I feel bad for what is going on or had happened in their life that drains them of the motivation or ability to exercise.

8 ปีที่แล้วdiscuss

Intelligence agencies generally avoid saying exactly why they suggest something and how they came to that conclusion. If agencies were fully open, adversaries would know what holes _they_ need to plug.

9 ปีที่แล้วdiscuss

From the second post,

> Alternatively, awk '{print $2}' netflix.tsv would have given us the same result. For this tutorial, I use cat to visually separate the input data from the AWK program itself. This also emphasizes that AWK can treat any input and not just existing files.

13 ปีที่แล้วdiscuss

Also, my numbering got a little off because it's late, and I can't edit. Looks funnier this way.