Hacker Newsnew | past | comments | ask | show | jobs | submit | sohei's commentslogin

Can anyone with more expertise comment on the use of cryptography.hazmat which is apparently (1) a frontend to openssl, (2) "full of land mines, dragons, and dinosaurs with laser guns"?


It's "hazmat" because cryptography is dangerous, and the library authors would strongly recommend you use an opinionated library that does the cryptography for you, like Nacl, where you don't think about which crypto primitives to use, but rather they're all set up for you. Here, they're working with an existing format and have specific requirements; they're doing the thing the "hazmat" library exists to facilitate.


How did you gather the comment histories? Would you mind sharing a copy?


See description at the bottom. We used the Hacker News API to pull data into BigQuery.

From there we ran them through an embedding model and indexed the embeddings in Pinecone.

The actual similarity search is done with Pinecone. (https://www.pinecone.io)


Using Google BigQuery is one way. This comment might be of use:

https://news.ycombinator.com/item?id=25075318

> A reminder that BigQuery (as used in the query in this link) is the best way to play with Hacker News data; don't scrape HN data manually! The `bigquery-public-data.hacker_news.full` table appears to be up to date with the most recent HN data as well (table last updated today). However, I'm not 100% sure the query is correct for unilaterally getting all links, as running the query on the full dataset returns the same results as running it from 2006-2015. And I value my sanity enough to not fuss around with the regex.


This is the story people tell in business schools and economics departments, but there's more to it than that.

It's also a way to sterilize new issuance to employees, primarily senior management. In a very real sense, it can be a backdoor payment to management. Issue shares to management. Buy shares on the market. Net effect is cash to management.

If buybacks are funded by debt issuance, it's also a way to lever the company. Remember Apple and Carl Icahn? That particular episode looks a lot like greenmail. Apple bought back shares, mainly from Icahn, so that Icahn would go away. The buybacks weren't exclusively from Icahn, so it wasn't technically greenmail.


The factual basis of your assertion is absolutely true, but your attitude is unhelpful and defeatist.

There is a chasm between "a state actor throws an 0day at you" and "Google remotely installs an app on your phone". The latter is done at scale. The former is expensive, risky, and used relatively rarely.

If you're organizing a protest movement, it's totally reasonable to factor government 0days into your threat model. For more boring people, running GrapheneOS is a great way to reduce the attack surface they expose to the advertising and mass surveillance industrial complex.


its not exactly a 0day if the ISP is communicating (through an intermediary) to a card the ISP gave you, that's just normal, unexpensive

And this is like, literally a state actor installing an app in this case?


In this case it requires the presence of Google Play Services. I'm unsure if there's any evidence that they can install apps without it being present.


A corollary of your question. If Google can lawfully install arbitrary apps on ordinary users' phones, can it also run arbitrary code on the personal devices of government officials investigating it for price fixing in the ad market?


"Arbitrary" is doing a lot of sneaky work here. You're implying that the law would somehow allow Google to manipulate investigators. But the law has broad allowances and exceptions in lots of areas, and competing permissions/denials that together weave specific allowances. There's little reason to think that the law couldn't allow app installation in general and also disallow either targeting of individuals or collection/manipulation of certain kinds of data.


>You're implying that the law would somehow allow Google to manipulate investigators.

Not the law. Google having root access on 2.5 billion android devices.

The law didn't allow Uber to greyball either. It did though.

This is a risk Google fully recognizes - it's why Google prevents f droid from updating apps one by one without user input. That's a privilege reserved exclusively for google play services.


Another question worth asking is "what is the governing law?" It is almost certainly contract law via Google's ToS. Government phones probably have different ToS, but government employee' personal phones have the same ToS we have.

If Google is asserting non-contractual rights, I'd like to know what they are.

Edit: I edited this comment because it was rude, and that was not my intent.


[Edit: the comment originally said their question wasn't implying anything] Of course you're implying something. If nothing else, you're implying the one might imply the other, and that the implication is worth attention.

The governing law that would protect people is a lot of things, and ToS is the least of it. The Wiretap Act applies, for example.


> ToS is the least of it

I'm afraid I disagree. Google running code on your phone implies it believes you have consented to that. That consent was not given in the app store, so it must have come from the ToS.

Consent is an exception to virtually every protection that exists: Wiretap Act, state wiretapping laws, the CFAA, and state computer trespass laws. Remember, consent is the difference between a home invasion and a dinner party.

So it seems that Google would have to cook up a pretty implausible stopping principle to argue that whatever allows them to do this does not also enable the hypothetical I described above.


If you've got a stock Android device you've obviously consented to Google running some code, and even updating to add new code after you bought it. On the other hand, apps are restricted based on permissions, and Google bypassing that would belie a consent theory.

You're making out like code is code and there aren't already existing lines and stopping principles, which just isn't true on its face.


> If Google can lawfully install arbitrary apps on ordinary users' phones

Of the partners in this, I think that the source of authority waa almost certainly the other one. It’s not Google, but the State of Massachusetts, whose authority is likely involved.


That also means it's going to be hard to sue over this, because the courts where you might do so are part of the same Commonwealth that authorized the action. Even if you could get a case on the docket, they'd just say the magic words "sovereign immunity" and it would disappear.


> personal devices of government officials investigating it for price fixing

Anything in the name of "improving our services".


Someone should archive a copy for reversing. One comment says it has "permission to utilize all device functions".


It doesn’t. All it can do is request Google Play Services to enable distributed covid exposure notifications, which in turn means the app itself doesn’t even get bluetooth beacon data.

Your calculator app has more tracking than this.


One comment also says that it's a Communist conspiracy.

Over in reality, though, the only permissions it has are to use the internet and bluetooth: https://hastebin.com/yexoyuluzu.xml


I would imagine you can get location by getting in range of those edison beacons google is pushing.


They bundled bluetooth and location permissions not long ago. Claimed their users were too ill-informed to know the difference.


Are you sure about that? Previously they required the location permission to scan for nearby bluetooth devices (because of the obvious implications). Recenly they've split that into it's own separate permission. It that what you're thinking of?


Yes, that is it. I did notice it's not the case anymore, but do you know if they admitted error with the change, or gave any kind of motivation?

I think that in this context, this previous issue is relevant.


Were you just guessing their motivation before, when you wrote "Claimed their users were too ill-informed to know the difference."?


Not guessing no, but I distilled what they said unfavorably. My understanding was that they thought they could not communicate to their customers the complexity of how bluetooth can be used to infer information about location. But somehow this communications barrier meant that they thought it was better to expose GPS data too? - Hence why I feel justified treating them unfavorably in this case. It was just an absurd way to reason that they presented. It should have been obvious that it's better to protect as much data as possible when the user often has no choice but to enable bluetooth.


If it works like all the other covid tracking apps it just records randomized bluetooth beacons emitted by all the phones near you.

You can get location in a ton of ways, but I doubt the OS will let you without the proper permission.


> I was shocked to learn that my new employer has a very different job definition for ML Engineer than the one I was familiar with.

Job titles do not have consistent meaning across regions or industries. People use buzzwords and hype to recruit funding and talent.

You've learned an important lesson: use interviews to gather information about companies and teams. "Can you describe in broad strokes a typical project for this role?" is a perfectly reasonable question to ask a hiring manager.


This 1000%.

I was an ML engineer, and to me it was far more about data management/cleaning than nitty gritty algorithms improvements, but that's definitely not industry standard.

I've also found "Product Engineer" job listings to range from essentially just a backend engineer to a product designer who codes.


This. I would add that role titles also exist for negotiation purposes. For example, a new employee may prefer to be called "research engineer" instead of "ML DevOps" to open up future career opportunities, even if the work done is the same.


There is a lot of outright corruption [1] and malfeasance [2] in the surveillance tech acquisition processes. It's probably because that process is so secret. Brandeis was right, "sunlight is the best disinfectant".

The strategy seems to be, "run the process in secret and present the result as a fait accompli". It's working.

[1] https://www.justice.gov/usao-ndil/pr/former-redflex-ceo-plea...

[2] https://www.vice.com/en/article/k7exem/banjo-ai-company-utah...


Is there a Matrix design document at a similar level of abstraction to [1,2,3]?

I want to use it, but the client (Riot? Element?) scares me with all its features, and I'm not sure I have a working mental model about what information is shared with whom and under what circumstances.

Is there a "blessed" client that doesn't run in the browser? Written in Python or Java?

[1] https://www.chromium.org/chromium-os/chromiumos-design-docs/... [2] https://www.chromium.org/chromium-os/chromiumos-design-docs/... [3] https://www.chromium.org/chromium-os/chromiumos-design-docs/...


There is a spec[1] as well as many third parties clients[2]

[1]: https://spec.matrix.org/unstable/

[2]: https://matrix.org/clients/


For a nicely documented and clean Matrix client implementation in Python, see weechat-matrix[1] and the matrix-nio[2] library it's based on. There's also Mirage[3] which is also based on matrix-nio and is a GUI client.

[1]: https://github.com/poljar/weechat-matrix

[2]: https://github.com/poljar/matrix-nio

[3]: https://github.com/mirukana/mirage


Last I tried to read the spec (months? a couple of years? who can even remember these days), it kind of sucked. The allegedly separate server spec silently presumed knowledge of all of the client spec without any specific references, and the description of the merge strategy was organized as a sequence of forward(!) diffs of successive algorithm versions. I gave up.

Has it improved since?


The spec is ok, but you do have to bounce around a lot. There are still areas that need improvement for sure, but we’ve been using it for about a year now with some level of success in building a chat product.


What are you building? I am about to start on a similar project.


Matrix powers our live chat product at https://groovehq.com


> It’s equally unsurprising that the ambitious children of highly educated and prosperous families themselves pursue such a similar path and achieve similar outcomes.

That kind of misses the point. You want to train as many effective engineers and researchers as you can. The point of the article is that most Ivies can comfortably train more people, but doing so would reduce some of the scarcity "value" of an Ivy League degree. Who cares? The goal is building.

Claude Shannon and Kelly Johnson both got their start at Michigan. The Michigan model of "train as many engineers as you can without compromising substance" is a great. If the Ivies want to create artificial scarcity to avoid "diluting brand value", then their ability to access public subsidies should be curtailed. Give it to Michigan instead.


But are Ivy League universities really that much better at training people or is their success mostly due to skimming from the top of the applicant pool? If the latter, then it seems like those students would do just as well if they didn't exist and nothing's really lost or gained either way by their exclusivity.

If they have a magic recipe for creating success, why can't other universities copy it? Does it depend on scarce elite professors? In that case, they probably can't scale themselves up either.


There’s been research on this point and it supports your thesis. After adjusting for applicant scholastic aptitude, the additional benefit from attending an elite school is “generally indistinguishable from zero.”

Excerpt from article*: In November 2002, the Quarterly Journal of Economics published a landmark paper** by the economists Stacy Dale and Alan Krueger that reached a startling conclusion. For most students, the salary boost from going to a super-selective school is “generally indistinguishable from zero” after adjusting for student characteristics, such as test scores. In other words, if Mike and Drew have the same SAT scores and apply to the same colleges, but Mike gets into Harvard and Drew doesn’t, they can still expect to earn the same income throughout their careers. Despite Harvard’s international fame and energetic alumni outreach, somebody like Mike would not experience an observable “Harvard effect.” Dale and Krueger even found that the average SAT scores of all the schools a student applies to is a more powerful predictor of success than the school that student actually attends.

* - https://www.theatlantic.com/ideas/archive/2018/12/does-it-ma...

** - https://www.nber.org/papers/w7322


I wonder if a similar analysis could find whether there is a 'skull and bones effect'.


Their magic recipe is putting a bunch of smart, mostly rich, people in one place and then telling them they are part of an exclusive community. Then they give them access to the older successful members of the community and constantly remind them that once they are the older successful members they need to help out the recent grads.


Sounds a lot like Freemasonry, but unlike Feemasonry, it seems to be actually working as intended.


You could also use the formula, "rich, mostly smart".


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: