AI Megathread

Aria

@Aria said in AI Megathread:

I’m just sitting here watching this with a look of vague horror on my face

AI has no concept of this. You can ask it to “write secure code” or whatever, but it fundamentally doesn’t know how to do that. I sincerely hope it does not take a rash of Therac 25 level disasters to teach the software industry that lesson again.

Yeeep. I’m not in medical research anymore, but I used to work in the office that monitored clinical trials for massive university hospital system, including training clinical research coordinators on how to maintain documentation to standard.

You do not mess around with people’s lives, livelihoods, and life savings. If you break those, there’s really no coming back. I don’t understand why we have to keep learning this, but I guess some tech bro billionaire and all his investors that can’t actually follow along with what he’s saying need the money to upgrade their yachts.

Hobbie

I work in fintech. We are involved with some big cranky banks. The current AI push driven by our CEO is several GDPR breaches waiting to happen and my security guy is sitting there pulling his… actually he has no hair so I suppose he’s pulling his beard out! I’m right there with him, both on the frustration and lack of hair.

Devs, infra, solution design et al, we don’t get paid to write lines of code, we get paid to write the right lines of code. That’s why we have PRs and reviewing them is where all the productivity maybe-gained is being absolutely-lost.

I’m not even on the dev teams, I’m in infra, and even I’m copping it from product people trying to push code to my repos now. UGH.

Third Eye

@Hobbie
LOLOLOL. I also work at a Fintech, in compliance, and the direction on AI is incredibly schizophrenic. They’re like WE LOVE AI THE CEO WANTS MORE AI…but also don’t put ANY of your work into public AI, and probably not even the proprietary ChatGPT rip-off we have internally, because you’re probably gonna violate personally identifiable information agreements. I just don’t use it and am doing all right. There have been some nice internal robotics enhancements that’ve come out of this whole mess but none of them have given me any confidence this stuff will take my job anytime soon.

dvoraen

@Hobbie said in AI Megathread:

I work in fintech. We are involved with some big cranky banks. The current AI push driven by our CEO is several GDPR breaches waiting to happen and my security guy is sitting there pulling his… actually he has no hair so I suppose he’s pulling his beard out! I’m right there with him, both on the frustration and lack of hair.

Devs, infra, solution design et al, we don’t get paid to write lines of code, we get paid to write the right lines of code. That’s why we have PRs and reviewing them is where all the productivity maybe-gained is being absolutely-lost.

I’m not even on the dev teams, I’m in infra, and even I’m copping it from product people trying to push code to my repos now. UGH.

But don’t worry, the solution will be to train an “AI” to accept and deny the right PRs so that way it’ll eventually get it right and then you can work on more important things and let “AI” fill in the rest.

… Twenty years later, when maybe something marketed as “AI” learns enough to write proper code instead of parrot it. (And after “a few” lawsuits and payouts related to GDPR and other data leaks, company implosions all over the world, etc. etc.)

But I’m not going to hold my breath on this.

Pavel

@dvoraen And somehow someone still needs to know COBOL.

Faraday

@Pavel Which GenAI will almost certainly never be able to because there isn’t enough COBOL stuff out there for it to steal for training data.

Pavel

@Faraday All COBOL knowledge is held exclusively by two men, both called Steve. They’re not allowed to travel at the same time, to avoid the risk of all worldly knowledge of COBOL being lost in the same incident.

Rathenhope

@Hobbie As another person who works in fintech, I have been incredibly lucky on this score - I’m the most senior tech other than the CTO, and both of us are incredibly sceptical of LLMs, so while there has been the occasional push to Do More With AI we’ve been able to stand firm and not bring it in to general use.

And we both got vindicated this quarter when two potential clients (the biggest we’d have) both went “we consider any use of AI to be high risk and we don’t want client information anywhere near it” and we went “excellent that’s our philosophy too.”

That said I’ve been banned from talking about LLMs on our all-hands calls as it invariably turns into a 20 minute rant about why they are bad for our purposes and the calls are only mean to be 30 minutes long.

Faraday

This English professor asserting that em-dashes are the biggest tell-tale sign of her students using AI is what I’m talking about when I complain about people picking on the dashes.

hellfrog

bring back downvotes

Hobbie

@Rathenhope said in AI Megathread:

two potential clients (the biggest we’d have) both went “we consider any use of AI to be high risk and we don’t want client information anywhere near it”

Just casually send me the names of these clients so I can get our CEO to try and sell to them, I really want to see that metaphorical ice bucket tipped all over him when he hears that.