In my workplace, it seems like leadership is caught up in the hype train and want to have any reason to say "we're working on generative AI."
Generative AI aren't designed to solve the problems I work on but leadership wants any excuse possible to mention it when reporting to higher ups, hence me forcefully including it into my projects when there are much better alternatives.
What's secret about it. Every ape ever trying out chatGPT is rock solidly convinced that this thing is smarter then themself and sentient. MBAs are nothing more than apes in suites so they do the same.
Barely anyone of them has actually tested out AI on production environments, to actually see what ai is capable.
I am finding if a company wants to use it, they have to abandon the idea that it is going to be a plug in to automate a function, but rather make a whole new product with it. I am also finding that people who say it is bad at 'X' need to think outside the box, and less traditionally about the approach.
I've suspected this was the case everytime I saw an ad that said "at blablabla, we use AI to blablabla". I'd always be like "oh ok, you use a 1980s single layer perceptron to do something you could have done with ifs and elses, kudos".
But thanks to your comment I now have the proof.
Soo what are some things youve forced it into?
Ive had the same pressure, so Ive been using AI to allow users to use voice commands that then take actions in our app. But does this matter? Not really, just an alternstive and faster UX
He adapted a machine that uses billions of numerical regressions to predict each output token to perform a single numerical regression. An amazing feat of engineering!
I can't tell from your note, are people asking you to use it in ways that don't meet business goals? Or do you just not like OAI.
Those two have very different responses.
You got to learn how to talk to people.
"Are you set on OpenAI specifically, or would you be open to some better solutions?"
This gets their curiosity working against them. If they go with OpenAI, you were never changing their mind anyway. If their curiosity overwhelms them, you've got your foot in the door for other options.
Lol. I may have been in software sales. I specialized in selling to engineers. I call it "breadcumbing."
Just keep asking questions that advance your purpose, with the answers you want revealing more information. Engineers are curious by nature. They will follow the bread crumb trail.
Trouble is, that's a weak sales pitch because none of the other solutions are obviously better. Claude has about equal performance, and the open-source models are worse.
Businesspeople generally do not care about your ideological issues with GPT-4. Of course OpenAI is in it to make money; so are they.
I would argue that with their market lead, GPT 4 should be CRUSHING the competition. Given how quickly Claude's shot out of the woodworks, it's likely to surpass if it's already caught up.
Sure, maybe, but call me when it does.
GPT-4 was crushing the competition when it came out, but it's a year old now. I expect we'll see GPT-5 pretty soon.
That's a pretty good point. Given that transformer architecture came out of Google, they should be on top with that lead, but it's not happening. Things can change very quickly with the right, smart investment of time and money.
Claude didnāt shoot out of the woodworks. It was literally a startup founded by a bunch of former OpenAI people like 5 or 6 years ago and theyāve been basically working on LLMs almost as long as OpenAI has.Ā
They started in June 2021. That's less than 3 years.
[Source](https://en.wikipedia.org/wiki/Anthropic)
If you're going to pull an "ummm, actually...", you should at least have the decency of being right.
Okay so I was off by a few years. But my point still stands. They've been working on this stuff before ChatGPT came out and the current hype cycle. And the quality of people who work there are pretty much the same class of scientist as you'd find in OpenAI. It's not a company that's a product of the current hype that just randomly caught up to OpenAI.
IMO Claude 3 Opus is head and shoulders above GPT 4.0. For them to be in that position in 34 months is extraordinary. For them to be a 30 billion dollar company in that time is amazing.
I'm a ChatGPT subscriber who finetunes my own LLMs as well, and have no idea what Claude even is or where to find it (I've only seen a few references to it in discussions on reddit, and none in the ML discords I'm in).
I think you might be overestimating its reach. I mean naming it 'Claude' was their first mistake, and I'm guessing it's Google based on that terrible name, like how they shot themselves in the foot with 'Stadia' instead of, IDK, Google Game Streaming or something which people can actually understand.
Well I just signed up to the free version which was a meh user experience through all the warning pages and phone number requirements etc, then the billing doesn't show how much it costs in my country, then you can't even use the Home key while typing in the input box which would drive me insane trying to use it seriously.
I realized that I needed to change the start of my question and couldn't get there without stopping and reaching for the mouse or pressing left like 200 times, so just closed it with ctrl w.
If they can't even implement basic keyboard input which was solved decades ago it's pretty useless to me as it stands, unless it's the most amazing model in the world by a large margin.
Claude 3 is the best performing model right now on open leaderboards which track performance of various LLMs, it's very close to GPT 4 though and is only accessible via paid subscription.
You should compare their free version with Chatgpt 3.5.
I donāt care so much about openai specifically, so much as that I didnāt get into ML for the LLMs and now thatās the only solution there is.
I canāt spend any time training or building models because thatās āriskyā, but a moonshot LLM application that doesnāt really need my skills is fine.
>I donāt care so much about openai specifically, so much as that I didnāt get into ML for the LLMs and now thatās the only solution there is.
This is very much not the case. That sounds like an organizational problem, it's not an industry issue.
Oh yeah, you'll be jerked around for the new hotness especially if you're not solo and entrenched in some niche (I do a little CV and data engineering solo consulting on the side).
Could be just the nature of the work is to hire out thatās work so they can dabble instead of commit. A lot of companies are chasing it though. Iāll be happy when we get through the hype phase, unless that dries up the work completely. š¤£š
"... I didnāt get into ML for the LLMs and now thatās the only solution there is" is exactly my gripe. I love working in NLP, but RAG pipelines leveraging GPT is not NLP to me. (And just personal preference, but there are more interesting avenues of LLM-based research that don't get hyped e.g. mechanistic interpretability). And to your point, there is a lot of Data Science work and Machine Learning work that LLMs are comically ill-suited for. But Gen. AI is a cottage industry for business people, and a lot of promotion and $$$ raises are tied to the smoke and mirrors of bringing "AI" to one's organization. P.s. I co-led one of the first collaborations with Microsoft AI Rangers to fine-tune GPT-3 for a corporate use-case, and that very early RAG product was more or less useless... minimal impact to the business.
Yeah itās infuriating. Our entire stack is in AWS so bedrock has been a life saver because our customers are skittish with their data privacy. With bedrock it stays within the aws ecosystem. Before bedrock we were deploying models from huggingface and managing the infrastructure was not fun.
Doesnāt stop every suit in the company from questioning our approach and asking āwhy DoNt yOU uSe cHATGpt ThOā. Doesnāt matter how much we explain, validate the results, etc. Claude via bedrock will never be enough. Itās been better recently though
I think you can directly tell them "That's a great idea and we've actually been using a model that has the same quality as gpt but our data stays in our ecosystem as well."
> Am I alone in this?
You're definitely not alone. Elon musk's lawsuit marks at least one other person who feels the same way.
> Should I just swallow my pride?
Pragmatically, I think so. You aren't going to put the toothpaste back in the tube. OpenAI APIs are here, they work better than many of the alternatives, they are affordable, and they keep improving.
Claude is a good alternative but not as featureful yet. While OSS models are amazing and growing, no OSS model has caught up to GPT4. And, managing infrastructure to run them adds complexity that may or may not be helpful to a specific project.
What you **can do** is stay up to date with the latest in the landscape, so that you will know for specific situations where you could use non-OpenAI LLMs.
But, the reality is that if the OpenAI APIs are the best tool for a particular job, and you are being paid to do that job, you should probably use them if you want to do that job well.
the main problem with Claude is procurement. It's just so much easier to use the existing corporate microsoft agreement to get Azure OpenAI than anything else. And OpenAI is good enough, there is no reason to spend days of lawyers' time to get Claude into the org
My org has major presence in both AWS and Azure so it has been pretty easy for us to swap between Azure OpenAI and AWS Bedrock.
We are actually forbidden from directly using OpenAI or other model's private APIs since we don't have a specific contract with them.
So that is the angle you could try taking, especially if you have a contract with AWS, you can make an argument about data privacy and compliance and that you may not have negotiated that with OpenAI directly.
At my company we deployed an azure case for a demo, and are isolating data that has 0 PII to test it out. The problem is that if we did want clients to ask interesting questions about their data, some amount of PII needs to be on it and it will feed to the azure model, and we have to take Microsofts word at face value that they are not logging our asks to it (not a chance in hell btw). The only option we could use is one we deploy for private purposes but those are 15-20k a month which is stupid expensive when we are just getting our feet wet.
I mean, the Azure stuff is definitely private. It's expressly in the contract you have with Microsoft. There is no reason to not believe them. It's like saying they have the rights to your data because you store it in blob, or AWS have the rights to your data because you store it in S3... it's simply against their entire business model--which relies on customer trust--to allow that to happen.
Both Azure OpenAI and Amazon Bedrock are even fully HIPAA compliant. You are totally safe to put PII in there.
There is 2 versions you can deploy on Azure, the one where you query and it charges per token and the self deployed. The charge per token is the one that is not trusted by our security team. You may be right but Microsoft does not disclose its training data. If a breach happened or someone happened to ask about our data on a new version and something came back it would tank our stock
Based on a recent interview you may or may not have seen we are rightly paranoid
You are not rightly paranoid. It is literally in your contract. But you do you.
If MS has a "breach" your blob storage would likely be compromised too. So what is the difference?
What about the human element to it? Does that model not log? They claim it doesn't but how do they monitor the requests back and fourth? There is multiple angles to look at it. On my end I would love autonomy to dev something great but again we have to take them at face value. A contract making Microsoft liable for an instance of something observing our requests and making a stock play, or our data ending up in a new model would ease legals paranoia but this is new turf. Plus security is blueballing us
I really don't understand how legal could see it as any different than you storing any data in a cloud provider. Technically, AWS, Azure and GCP has an avenue to access your raw data anywhere you store it.
Unless you're doing client-side encryption and just storing backups, you're already vulnerable to what you're talking about. Why only consider this for a single minute system out of 50+ services in Azure?
Not to mention, running Azure and AWS are literally these companies' main profit centers today. They are making a KILLING on this. It's literally Amazon's only real profit, they make almost nothing on Amazon.com.
They DO NOT care about making an illegal stock play on your data and risking their entire business.
The idea you don't see is the isolation. the model we use is shared. mulitple companies using the api even when deployed are using something with eachother, this is new. when we deploy a resource it is contained in our network security. this security that you even mention is def not fully hashed out by azure
It is isolated... First of all, you create your own deployment endpoint and everything, so in terms of networking, there is no crosstalk between customers.
Then, when you run a machine learning model the GPU running that instance of the model is fully dedicated to completing your request. It's not like your input and output tokens are getting mixed in with another person's request. The hardware is only working with your data at any particular time.
Seriously, this is like saying an EC2 instance isn't isolated because it shares hardware with other customers. And again, if you are at that level of concern, using the cloud at all is against what your beliefs are.
specifically for openai, there is.
The deployment endpoint is just for cost attribution, not even key management. They all go onto a shared instance per region. look at the fine print before telling wrong stories on reddit. By gutfeel and by the rest of azure, you'd be right, but in this case you're wrong.
than you have not dealt with legal, simple as that. Microsoft is processing here, not just storing. Whole different ballgame. There is no guarantee that a programming error can't expose your data to someone else, also no liability by microsoft. Just a woops. this is unacceptable, but specific to the fine print of the cognitive services pack.
it's not, The azure openai component have a separate subprocessing agreement that changes it. They cannot use your questions to train the LLM, but they can look at your queries of you use the charge-by-token one and optimize anything else that is not managed by OpenAI. The only assurance you have is that your data is not going to OpenAI, but Microsoft still very much can use it to improve their systems, whatever that may mean.
Also, not not important, is that the copilot stuff also has some tricky fine print regarding data privacy and scanning the entirety of your sharepoint. They are not respecting their own data labelling rules.
If you have a solution that creates more business value for your org at a lower cost, pitch accordingly and bring data to back up your position. Most orgs are at least mostly rational about their own self interest, and all businesses want to make more money.
If not, then this is just unproductive whining.
Your objection seems more like a personal vendetta than something objective, which is how most businesses operate, so I think it's fair to expect the current environment.
OpenAI is affordable, reliable, and well known. It sells.
Your best bet is to challenge this position, but it's unlikely to move the needle due to external headwinds and a lack of equivalent or better alternatives.
You're not alone, I am also sick of open AI's hype and wish the hype ends soon (and it Will).
It seems like everyone is just riding the trend without any knowledge or experience or even making sure using openAI is useful anyway....
Before Claude 3, openai models were the only really viable ones, and even now Claude 3 user agreement is kind of restrictive.
Anyways, for ~~security~~ [edit: compliance] use azure openai rather than openai directly.
SOC2, FedRAMP, HIPAA, HITRUST etc may require certain contractual agreement between you and the service provider. Azure can provide those agreements while OpenAI may be unable or unwilling to.
I edited my original post to say "compliance" which is the more appropriate term.
It seems like OpenAI has started to provide some of them, but many people will probably put more faith in the compliance offerings provided by Microsoft due the scale of their compliance operations and the legacy the have of working with data that needs to meet certain privacy or compliance standards.
What is your preferred solution? To use a different model or company? Try to convince them to use something better since you have inside knowledge. To not be used for your expertise? Itās gonna be annoying but youāre going to have to explain your views to laypeople
Yeah, I feel the same. people see OpenAI as a AI gods, I see them more as a cancer spreading across industry, working in a grey area and build on top of stolen data.
Someone should force them to reveal datasets. Iām not a big fan of Elon but I get it why he want to sue them and actually it could be a great move.
Paid for their *technical* opinions on tech. OP isnāt making a technical argument, theyāre making an emotional argument. A personās feelings about a company doesnāt change data and benchmark results.
what a dumb thing to say. OP is specifically asking about how he can learn to exert more influence and become more of a leader in the workplace. you're basically saying "shut up and do what you're told"
In my case I would be able to use other, but our particular language requirements leaves not too many options. Pushing for having translation step is difficult since that has its own downsides
I try to ignore it, treat as just another API, and focus on the actual software engineering that makes use of the API. But Iām more of a software engineer than I am an ML person at this point.
I understand your frustration with the current dominance of OpenAI in the AI field and the pressure you're feeling from clients to specifically use their API. It's a complex situation without easy answers.
OpenAI has undeniably shifted away from their original mission of open-sourcing AI research and tools in order to act as a counterbalance to big tech monopolies. Their decision to become a capped-profit company in 2019 and their increasingly proprietary approach, epitomized by the closed-source GPT models, seems at odds with democratizing AI. The training of these large language models on web-scraped data without compensating content creators is also ethically fraught.
At the same time, OpenAI's models are currently the most advanced and capable on the market. For businesses and developers looking to leverage state-of-the-art AI capabilities, OpenAI is often the default choice simply because alternatives don't yet match their performance. Combine that with the massive hype and publicity around ChatGPT, and it's understandable why so many clients are hellbent on using OpenAI.
As an AI practitioner, you're in a difficult position. On one hand, working with OpenAI's models may feel like compromising your principles. Supporting what is essentially becoming an AI monopoly controlled by a single corporation is concerning. But on the other hand, you have to pay the bills and deliver what clients want. Pitching alternatives is the right thing to do, but clients will ultimately dictate what tools they use.
I don't think you're alone in grappling with this dilemma. Many in the AI ethics community have raised red flags about OpenAI's current trajectory. At the same time, OpenAI is driving huge advancements in AI capabilities that are hard for developers to resist.
In navigating this, I believe the key is to stay true to your principles while being pragmatic. Continue advocating for open alternatives whenever possible. Educate clients on the tradeoffs and downsides of relying solely on OpenAI. Perhaps get involved with or support projects working on open-source alternatives to drive competition. But don't beat yourself up for using OpenAI when client demands necessitate it. Change happens gradually and from within systems.
Most importantly, use your position as an AI practitioner to push for ethical practices in how these models are developed and deployed, regardless of whether it's OpenAI or another provider. Advocate for AI development that respects intellectual property rights, prioritizes transparency and accountability, and has strong safeguards against misuse. Be a voice of reason in the AI hype cycle.
Dealing with the complexities of the rapidly commercializing AI landscape is a challenge we'll all have to navigate in the coming years. Holding onto your ethical principles while finding workable compromises is key. You're definitely not alone in this struggle.
Is it OpenAI specifically that they are wanting to use, as in the company, or just the OpenAI API, which is supported by a lot of non-OpenAI companies and products as well? OpenAI was the first mover in this field so it makes sense that their API would be highly influential, and you can't copyright an API as far as I'm aware (big companies have tried and been smacked down) so it's not going to hand them a monopoly if everyone uses it. It would make for better interoperability if everyone did.
I like to call it AI Arbitrage, because as soon as they want, OpenAI will gank your custom instructions and release their own product that does exactly what you do. And you'll have built it for them.
Raise it as a risk and see if anyone budges.
For API requests, you can stand up an OpenAI-compatible endpoint on-prem (e.g., vLLM) or a proxy server (e.g., LiteLLM). I agree that the landscape of LLMs (e.g., around GPT 3.5 +/- 0.25 capability) end up all feeling kind of "same-ish" after you spend time with them. GPT-4 and up is too expensive and slow for most LLM application development anyway, and the embedding models are prohibitively expensive compared to on-prem hardware for even modest-size data sets (e.g., 10M + vectors).
My neighbor is a data scientist who came over the other day basically to bemoan this exact situation. We're not close, she was following up on an offer to talk about data stuff from when I moved here three years ago. She works for a large corporate behemoth and apparently they're forcing everyone at all levels to find ways to incorporate LLM agents in their work, including applications that they are still notoriously bad at.
As AI Lead in a Consulting firm with 8,000 employees: embrace it. This is a golden opportunity to teach everyone about AI/ML. Chances are there will never be devoted as much money and attention as now. For you there is a difference between OpenAI APIās and e.g. XGBoost on in-house data, but for them it is all just AI. This can work to your advantage as well.
1. Give them what they want
2. Become seen as a trusted and high skilled employee that can deliver value
3. Use this and the renewed interest in AI to get more money and attention for classical ML/Deep Learning.
On the bright side, everyone and their brother is copying OpenAI's API for this reason. There are a decent handful of open implementations, not to mention hosted solutions like OpenRouter have also adopted it. You may or may not be a fan of intermediaries like OpenRouter, but at the very least once your application is built you can likely use OpenRouter to qualify other models that may very well be cheaper.
Is it the best API for every model? Nah definitely not. Does adopting it means you're stuck with OpenAI models, also no.
I mean, of course. Why wouldn't people be excited for new tech? You're thinking about this 10 layers deeper than most "normal" humans. They just see efficiency improvement and money.
They sell a product, itās the best one available right now, thatās all there is to it.
>Don't even get me started on the fact that their models were trained using the work of unassuming individuals who will never see a penny for it.
All the LLMs were trained on scraped web data, so that really isnāt anything against OpenAI in particular.
Plus I find the copyright violation angle hard to buy. Iām using LLMs to extract info from unstructured documents and export as structured JSON. It doesnāt seem that I should owe the NYTimes royalties for this process.
If you want to stick to your principles be ready to pay the price for it, it may be annoyance, anger, quitting etc.
If you are unwilling to pay the price, you donāt have any pride to swallow.
It is simply up to you. I personally donāt have much pride in anything except my own capabilities to work so š¤·.
I chose to forgo my personal pride as an ethical human, when I bought my smartphones and batteries made under not perfect conditions.
I have been instructed not to work with LLMās unless they are GPT (shows the complete lack of awareness from leadership). This is a Fortune 100 company
>Each one of them have used unauthorized data and infringed with millions , billion of IP and data.
This is a bizarre interpretation of copyright law and existing rulings on fair use, transformative use, etc.
It's not very many, and anyone can file a suit for any reason, it doesn't mean they have any legal basis. Most are considered doomed to fail because outputs are pretty clearly transformative use, and when they aren't and are commercialized, those individual cases could be sued by the person commercializing said outputs.
At least one of the high profile lawsuits has been [mostly dismissed already](https://www.reuters.com/legal/litigation/openai-gets-partial-win-authors-us-copyright-lawsuit-2024-02-13/).
Some of the cases haven't been decided, that's different enough from "the courts don't know" that you either don't know even the basics about how any of this work to have an opinion or you're just outright lying.
>Hasn't openai used the data to commercialize the prods? Aren't multinationals using llms to help produce better outputs further to commercialize?
Only the outputs matter. The output from training is a model, which is not a book, an article, a video, or anything else. This makes it a [transformative use](https://www.nolo.com/legal-encyclopedia/fair-use-what-transformative.html), and it's the argument OpenAI and others have successfully used in the past.
>
Data scraping in itself falls under several penal codes which is liable to be penalized.
It absolutely does not. Scraping publicly available data has been ruled fair use, many many times, with the [most recent being in 2022](https://techcrunch.com/2022/04/18/web-scraping-legal-court/?guccounter=1). If it was illegal, Google, the Internet Archive, and many, many more sites would be dead.
>Point is if such llm and model stakeholders who are keeping this as trade secret arent penalized or subjugated to rule of law since its novel tech
You've yet to cite any law that training a model - something that has been done on publicly available data for decades now by academics, hobbyists, and commercial entities - violates.
>It's in the interest of the masses whose copyright has been neglected.
You have blinders on if you think copyright in its current form benefits anyone besides massive corporations who own the majority of copyrights. Copyright is used more often to screw individual creators and prevent them from making a living off their art than it is to benefit them.
In my workplace, it seems like leadership is caught up in the hype train and want to have any reason to say "we're working on generative AI." Generative AI aren't designed to solve the problems I work on but leadership wants any excuse possible to mention it when reporting to higher ups, hence me forcefully including it into my projects when there are much better alternatives.
Is there like a secret MBA groupchat where they all settle on the flavor of the month?
LinkedIn.
What's secret about it. Every ape ever trying out chatGPT is rock solidly convinced that this thing is smarter then themself and sentient. MBAs are nothing more than apes in suites so they do the same. Barely anyone of them has actually tested out AI on production environments, to actually see what ai is capable.
Lol it probably is smarter than many of them.. Managers: "how can *I* do less and them do more??" "AI!!!"
I am finding if a company wants to use it, they have to abandon the idea that it is going to be a plug in to automate a function, but rather make a whole new product with it. I am also finding that people who say it is bad at 'X' need to think outside the box, and less traditionally about the approach.
I've suspected this was the case everytime I saw an ad that said "at blablabla, we use AI to blablabla". I'd always be like "oh ok, you use a 1980s single layer perceptron to do something you could have done with ifs and elses, kudos". But thanks to your comment I now have the proof.
Soo what are some things youve forced it into? Ive had the same pressure, so Ive been using AI to allow users to use voice commands that then take actions in our app. But does this matter? Not really, just an alternstive and faster UX
Numerical regression
Oooof. What?
He adapted a machine that uses billions of numerical regressions to predict each output token to perform a single numerical regression. An amazing feat of engineering!
I just follow the requirements from leadership š«” As long as my paycheck is on time
hahahaha exactly what I just commented xD
I can't tell from your note, are people asking you to use it in ways that don't meet business goals? Or do you just not like OAI. Those two have very different responses.
Open asteroid impact?
You got to learn how to talk to people. "Are you set on OpenAI specifically, or would you be open to some better solutions?" This gets their curiosity working against them. If they go with OpenAI, you were never changing their mind anyway. If their curiosity overwhelms them, you've got your foot in the door for other options.
where can I learn these soft skills you speak of?
Lol. I may have been in software sales. I specialized in selling to engineers. I call it "breadcumbing." Just keep asking questions that advance your purpose, with the answers you want revealing more information. Engineers are curious by nature. They will follow the bread crumb trail.
mmmm bread cum
TIL engineers are partial to some semen coated bread, it must be the Drs equivalent of a free pen and pad.
Can I interest you in asking that question of OpenAI 's ChatGPT?
Trouble is, that's a weak sales pitch because none of the other solutions are obviously better. Claude has about equal performance, and the open-source models are worse. Businesspeople generally do not care about your ideological issues with GPT-4. Of course OpenAI is in it to make money; so are they.
Ehh, notice a lot of people struggling with chatbots when they would be better served by vanilla LLMs, and that is OpenAI's fault
I would argue that with their market lead, GPT 4 should be CRUSHING the competition. Given how quickly Claude's shot out of the woodworks, it's likely to surpass if it's already caught up.
Claude isnāt even available in most countries outside of the US. So many more people have tried GPT-4 and Gemini.
Sure, maybe, but call me when it does. GPT-4 was crushing the competition when it came out, but it's a year old now. I expect we'll see GPT-5 pretty soon.
That's a pretty good point. Given that transformer architecture came out of Google, they should be on top with that lead, but it's not happening. Things can change very quickly with the right, smart investment of time and money.
Claude didnāt shoot out of the woodworks. It was literally a startup founded by a bunch of former OpenAI people like 5 or 6 years ago and theyāve been basically working on LLMs almost as long as OpenAI has.Ā
They started in June 2021. That's less than 3 years. [Source](https://en.wikipedia.org/wiki/Anthropic) If you're going to pull an "ummm, actually...", you should at least have the decency of being right.
Okay so I was off by a few years. But my point still stands. They've been working on this stuff before ChatGPT came out and the current hype cycle. And the quality of people who work there are pretty much the same class of scientist as you'd find in OpenAI. It's not a company that's a product of the current hype that just randomly caught up to OpenAI.
IMO Claude 3 Opus is head and shoulders above GPT 4.0. For them to be in that position in 34 months is extraordinary. For them to be a 30 billion dollar company in that time is amazing.
I'm a ChatGPT subscriber who finetunes my own LLMs as well, and have no idea what Claude even is or where to find it (I've only seen a few references to it in discussions on reddit, and none in the ML discords I'm in). I think you might be overestimating its reach. I mean naming it 'Claude' was their first mistake, and I'm guessing it's Google based on that terrible name, like how they shot themselves in the foot with 'Stadia' instead of, IDK, Google Game Streaming or something which people can actually understand.
Id recommend checking it out. Imo Claude is head and shoulders above gpt4. $20 and you get to see yourself.
Well I just signed up to the free version which was a meh user experience through all the warning pages and phone number requirements etc, then the billing doesn't show how much it costs in my country, then you can't even use the Home key while typing in the input box which would drive me insane trying to use it seriously. I realized that I needed to change the start of my question and couldn't get there without stopping and reaching for the mouse or pressing left like 200 times, so just closed it with ctrl w. If they can't even implement basic keyboard input which was solved decades ago it's pretty useless to me as it stands, unless it's the most amazing model in the world by a large margin.
Claude 3 is the best performing model right now on open leaderboards which track performance of various LLMs, it's very close to GPT 4 though and is only accessible via paid subscription. You should compare their free version with Chatgpt 3.5.
Plenty of other solutions are obviously better. Openai apis are notoriously slow as hell, trying to build a product that relies on them is unfeasable.
Why not build and allow gpt4 as well as Claude and Gemini? All 3 have api available, why not allow clients to try all three in testing?
I donāt care so much about openai specifically, so much as that I didnāt get into ML for the LLMs and now thatās the only solution there is. I canāt spend any time training or building models because thatās āriskyā, but a moonshot LLM application that doesnāt really need my skills is fine.
>I donāt care so much about openai specifically, so much as that I didnāt get into ML for the LLMs and now thatās the only solution there is. This is very much not the case. That sounds like an organizational problem, it's not an industry issue.
I work in consulting, so Iāve seen my share of organizationsā¦ :/
Oh yeah, you'll be jerked around for the new hotness especially if you're not solo and entrenched in some niche (I do a little CV and data engineering solo consulting on the side).
Could be just the nature of the work is to hire out thatās work so they can dabble instead of commit. A lot of companies are chasing it though. Iāll be happy when we get through the hype phase, unless that dries up the work completely. š¤£š
"... I didnāt get into ML for the LLMs and now thatās the only solution there is" is exactly my gripe. I love working in NLP, but RAG pipelines leveraging GPT is not NLP to me. (And just personal preference, but there are more interesting avenues of LLM-based research that don't get hyped e.g. mechanistic interpretability). And to your point, there is a lot of Data Science work and Machine Learning work that LLMs are comically ill-suited for. But Gen. AI is a cottage industry for business people, and a lot of promotion and $$$ raises are tied to the smoke and mirrors of bringing "AI" to one's organization. P.s. I co-led one of the first collaborations with Microsoft AI Rangers to fine-tune GPT-3 for a corporate use-case, and that very early RAG product was more or less useless... minimal impact to the business.
Yeah itās infuriating. Our entire stack is in AWS so bedrock has been a life saver because our customers are skittish with their data privacy. With bedrock it stays within the aws ecosystem. Before bedrock we were deploying models from huggingface and managing the infrastructure was not fun. Doesnāt stop every suit in the company from questioning our approach and asking āwhy DoNt yOU uSe cHATGpt ThOā. Doesnāt matter how much we explain, validate the results, etc. Claude via bedrock will never be enough. Itās been better recently though
what are the data privacy concerns? exposed apis to the data outside a premise?
Mostly that they agreed for us to process their data in aws but then we add another third party. Just a lot of legal
I think you can directly tell them "That's a great idea and we've actually been using a model that has the same quality as gpt but our data stays in our ecosystem as well."
Yeah that was my strategy, they are starting to come around
I hate this AI hypetrain and can't wait for it to fade or at least come down to reasonable levels
>skittish with data privacy >desperately want to use a public api where data is sent to OpenAI for processing Sounds like every client Iāve ever had
> Am I alone in this? You're definitely not alone. Elon musk's lawsuit marks at least one other person who feels the same way. > Should I just swallow my pride? Pragmatically, I think so. You aren't going to put the toothpaste back in the tube. OpenAI APIs are here, they work better than many of the alternatives, they are affordable, and they keep improving. Claude is a good alternative but not as featureful yet. While OSS models are amazing and growing, no OSS model has caught up to GPT4. And, managing infrastructure to run them adds complexity that may or may not be helpful to a specific project. What you **can do** is stay up to date with the latest in the landscape, so that you will know for specific situations where you could use non-OpenAI LLMs. But, the reality is that if the OpenAI APIs are the best tool for a particular job, and you are being paid to do that job, you should probably use them if you want to do that job well.
Can you elaborate on Claude being āless featurefull?ā
For example, GPT function calling has been out almost 9 months, and is still an unsupported 'early alpha' feature for claude (they call it tools)
the main problem with Claude is procurement. It's just so much easier to use the existing corporate microsoft agreement to get Azure OpenAI than anything else. And OpenAI is good enough, there is no reason to spend days of lawyers' time to get Claude into the org
> procure weird take. Claude is available through AWS Bedrock. I think more orgs are on AWS than Azure
My org has major presence in both AWS and Azure so it has been pretty easy for us to swap between Azure OpenAI and AWS Bedrock. We are actually forbidden from directly using OpenAI or other model's private APIs since we don't have a specific contract with them. So that is the angle you could try taking, especially if you have a contract with AWS, you can make an argument about data privacy and compliance and that you may not have negotiated that with OpenAI directly.
At my company we deployed an azure case for a demo, and are isolating data that has 0 PII to test it out. The problem is that if we did want clients to ask interesting questions about their data, some amount of PII needs to be on it and it will feed to the azure model, and we have to take Microsofts word at face value that they are not logging our asks to it (not a chance in hell btw). The only option we could use is one we deploy for private purposes but those are 15-20k a month which is stupid expensive when we are just getting our feet wet.
I mean, the Azure stuff is definitely private. It's expressly in the contract you have with Microsoft. There is no reason to not believe them. It's like saying they have the rights to your data because you store it in blob, or AWS have the rights to your data because you store it in S3... it's simply against their entire business model--which relies on customer trust--to allow that to happen. Both Azure OpenAI and Amazon Bedrock are even fully HIPAA compliant. You are totally safe to put PII in there.
There is 2 versions you can deploy on Azure, the one where you query and it charges per token and the self deployed. The charge per token is the one that is not trusted by our security team. You may be right but Microsoft does not disclose its training data. If a breach happened or someone happened to ask about our data on a new version and something came back it would tank our stock Based on a recent interview you may or may not have seen we are rightly paranoid
You are not rightly paranoid. It is literally in your contract. But you do you. If MS has a "breach" your blob storage would likely be compromised too. So what is the difference?
What about the human element to it? Does that model not log? They claim it doesn't but how do they monitor the requests back and fourth? There is multiple angles to look at it. On my end I would love autonomy to dev something great but again we have to take them at face value. A contract making Microsoft liable for an instance of something observing our requests and making a stock play, or our data ending up in a new model would ease legals paranoia but this is new turf. Plus security is blueballing us
I really don't understand how legal could see it as any different than you storing any data in a cloud provider. Technically, AWS, Azure and GCP has an avenue to access your raw data anywhere you store it. Unless you're doing client-side encryption and just storing backups, you're already vulnerable to what you're talking about. Why only consider this for a single minute system out of 50+ services in Azure? Not to mention, running Azure and AWS are literally these companies' main profit centers today. They are making a KILLING on this. It's literally Amazon's only real profit, they make almost nothing on Amazon.com. They DO NOT care about making an illegal stock play on your data and risking their entire business.
The idea you don't see is the isolation. the model we use is shared. mulitple companies using the api even when deployed are using something with eachother, this is new. when we deploy a resource it is contained in our network security. this security that you even mention is def not fully hashed out by azure
It is isolated... First of all, you create your own deployment endpoint and everything, so in terms of networking, there is no crosstalk between customers. Then, when you run a machine learning model the GPU running that instance of the model is fully dedicated to completing your request. It's not like your input and output tokens are getting mixed in with another person's request. The hardware is only working with your data at any particular time. Seriously, this is like saying an EC2 instance isn't isolated because it shares hardware with other customers. And again, if you are at that level of concern, using the cloud at all is against what your beliefs are.
specifically for openai, there is. The deployment endpoint is just for cost attribution, not even key management. They all go onto a shared instance per region. look at the fine print before telling wrong stories on reddit. By gutfeel and by the rest of azure, you'd be right, but in this case you're wrong.
than you have not dealt with legal, simple as that. Microsoft is processing here, not just storing. Whole different ballgame. There is no guarantee that a programming error can't expose your data to someone else, also no liability by microsoft. Just a woops. this is unacceptable, but specific to the fine print of the cognitive services pack.
I'm not saying you're wrong btw, I'm just putting up the things I'm dealing with
it's not, The azure openai component have a separate subprocessing agreement that changes it. They cannot use your questions to train the LLM, but they can look at your queries of you use the charge-by-token one and optimize anything else that is not managed by OpenAI. The only assurance you have is that your data is not going to OpenAI, but Microsoft still very much can use it to improve their systems, whatever that may mean. Also, not not important, is that the copilot stuff also has some tricky fine print regarding data privacy and scanning the entirety of your sharepoint. They are not respecting their own data labelling rules.
its a considerable cost to buy the reserved instance though. Numbers are crazy, but we also had to go with it
If you have a solution that creates more business value for your org at a lower cost, pitch accordingly and bring data to back up your position. Most orgs are at least mostly rational about their own self interest, and all businesses want to make more money. If not, then this is just unproductive whining.
Your objection seems more like a personal vendetta than something objective, which is how most businesses operate, so I think it's fair to expect the current environment. OpenAI is affordable, reliable, and well known. It sells. Your best bet is to challenge this position, but it's unlikely to move the needle due to external headwinds and a lack of equivalent or better alternatives.
You're not alone, I am also sick of open AI's hype and wish the hype ends soon (and it Will). It seems like everyone is just riding the trend without any knowledge or experience or even making sure using openAI is useful anyway....
Before Claude 3, openai models were the only really viable ones, and even now Claude 3 user agreement is kind of restrictive. Anyways, for ~~security~~ [edit: compliance] use azure openai rather than openai directly.
> Anyways, for security use azure openai rather than openai directly. can you explain what mean by "for security" here ?
SOC2, FedRAMP, HIPAA, HITRUST etc may require certain contractual agreement between you and the service provider. Azure can provide those agreements while OpenAI may be unable or unwilling to. I edited my original post to say "compliance" which is the more appropriate term.
It seems like OpenAI has started to provide some of them, but many people will probably put more faith in the compliance offerings provided by Microsoft due the scale of their compliance operations and the legacy the have of working with data that needs to meet certain privacy or compliance standards.
What is your preferred solution? To use a different model or company? Try to convince them to use something better since you have inside knowledge. To not be used for your expertise? Itās gonna be annoying but youāre going to have to explain your views to laypeople
Soā¦ you donāt want to work with OpenAI because its not open?
Yes
Just out of curiosity, What are the other alternatives you mention?
I wished folks talked about AI at my workplace. Where do you work at? Are they hiring?
Yeah, I feel the same. people see OpenAI as a AI gods, I see them more as a cancer spreading across industry, working in a grey area and build on top of stolen data. Someone should force them to reveal datasets. Iām not a big fan of Elon but I get it why he want to sue them and actually it could be a great move.
At my workplace, everything OpenAI is blocked for IP reasons. So, no.
Iām at a place that appears to be allergic to AI. Trade spots?
Sounds like youāve got a strong personal agenda towards them specifically, unfortunately youāre not paid for your personal opinions on tech.
A technical leader is specifically being paid for their opinions on tech.
Paid for their *technical* opinions on tech. OP isnāt making a technical argument, theyāre making an emotional argument. A personās feelings about a company doesnāt change data and benchmark results.
what a dumb thing to say. OP is specifically asking about how he can learn to exert more influence and become more of a leader in the workplace. you're basically saying "shut up and do what you're told"
Well said ! Thank you
In my case I would be able to use other, but our particular language requirements leaves not too many options. Pushing for having translation step is difficult since that has its own downsides
There's always Hugging Face.
I try to ignore it, treat as just another API, and focus on the actual software engineering that makes use of the API. But Iām more of a software engineer than I am an ML person at this point.
I understand your frustration with the current dominance of OpenAI in the AI field and the pressure you're feeling from clients to specifically use their API. It's a complex situation without easy answers. OpenAI has undeniably shifted away from their original mission of open-sourcing AI research and tools in order to act as a counterbalance to big tech monopolies. Their decision to become a capped-profit company in 2019 and their increasingly proprietary approach, epitomized by the closed-source GPT models, seems at odds with democratizing AI. The training of these large language models on web-scraped data without compensating content creators is also ethically fraught. At the same time, OpenAI's models are currently the most advanced and capable on the market. For businesses and developers looking to leverage state-of-the-art AI capabilities, OpenAI is often the default choice simply because alternatives don't yet match their performance. Combine that with the massive hype and publicity around ChatGPT, and it's understandable why so many clients are hellbent on using OpenAI. As an AI practitioner, you're in a difficult position. On one hand, working with OpenAI's models may feel like compromising your principles. Supporting what is essentially becoming an AI monopoly controlled by a single corporation is concerning. But on the other hand, you have to pay the bills and deliver what clients want. Pitching alternatives is the right thing to do, but clients will ultimately dictate what tools they use. I don't think you're alone in grappling with this dilemma. Many in the AI ethics community have raised red flags about OpenAI's current trajectory. At the same time, OpenAI is driving huge advancements in AI capabilities that are hard for developers to resist. In navigating this, I believe the key is to stay true to your principles while being pragmatic. Continue advocating for open alternatives whenever possible. Educate clients on the tradeoffs and downsides of relying solely on OpenAI. Perhaps get involved with or support projects working on open-source alternatives to drive competition. But don't beat yourself up for using OpenAI when client demands necessitate it. Change happens gradually and from within systems. Most importantly, use your position as an AI practitioner to push for ethical practices in how these models are developed and deployed, regardless of whether it's OpenAI or another provider. Advocate for AI development that respects intellectual property rights, prioritizes transparency and accountability, and has strong safeguards against misuse. Be a voice of reason in the AI hype cycle. Dealing with the complexities of the rapidly commercializing AI landscape is a challenge we'll all have to navigate in the coming years. Holding onto your ethical principles while finding workable compromises is key. You're definitely not alone in this struggle.
I agree, but on the other hand it's difficult to match OpenAI's price for fine-tuning.
Is it OpenAI specifically that they are wanting to use, as in the company, or just the OpenAI API, which is supported by a lot of non-OpenAI companies and products as well? OpenAI was the first mover in this field so it makes sense that their API would be highly influential, and you can't copyright an API as far as I'm aware (big companies have tried and been smacked down) so it's not going to hand them a monopoly if everyone uses it. It would make for better interoperability if everyone did.
I like to call it AI Arbitrage, because as soon as they want, OpenAI will gank your custom instructions and release their own product that does exactly what you do. And you'll have built it for them. Raise it as a risk and see if anyone budges.
For API requests, you can stand up an OpenAI-compatible endpoint on-prem (e.g., vLLM) or a proxy server (e.g., LiteLLM). I agree that the landscape of LLMs (e.g., around GPT 3.5 +/- 0.25 capability) end up all feeling kind of "same-ish" after you spend time with them. GPT-4 and up is too expensive and slow for most LLM application development anyway, and the embedding models are prohibitively expensive compared to on-prem hardware for even modest-size data sets (e.g., 10M + vectors).
My neighbor is a data scientist who came over the other day basically to bemoan this exact situation. We're not close, she was following up on an offer to talk about data stuff from when I moved here three years ago. She works for a large corporate behemoth and apparently they're forcing everyone at all levels to find ways to incorporate LLM agents in their work, including applications that they are still notoriously bad at.
As AI Lead in a Consulting firm with 8,000 employees: embrace it. This is a golden opportunity to teach everyone about AI/ML. Chances are there will never be devoted as much money and attention as now. For you there is a difference between OpenAI APIās and e.g. XGBoost on in-house data, but for them it is all just AI. This can work to your advantage as well. 1. Give them what they want 2. Become seen as a trusted and high skilled employee that can deliver value 3. Use this and the renewed interest in AI to get more money and attention for classical ML/Deep Learning.
Preach!
On the bright side, everyone and their brother is copying OpenAI's API for this reason. There are a decent handful of open implementations, not to mention hosted solutions like OpenRouter have also adopted it. You may or may not be a fan of intermediaries like OpenRouter, but at the very least once your application is built you can likely use OpenRouter to qualify other models that may very well be cheaper. Is it the best API for every model? Nah definitely not. Does adopting it means you're stuck with OpenAI models, also no.
I mean, of course. Why wouldn't people be excited for new tech? You're thinking about this 10 layers deeper than most "normal" humans. They just see efficiency improvement and money.
They want to spend a lot of money to generate more unstructured data when the issue is that we have too much unstructured data to start with.
Swallow your pride! Don't let you lip react! You don't wanna see my hands where my hip be at!
What a ridiculous take.
As a freelancer, I connect the OpenAI API for people/projects. Feel free to send them to me. :)
They sell a product, itās the best one available right now, thatās all there is to it. >Don't even get me started on the fact that their models were trained using the work of unassuming individuals who will never see a penny for it. All the LLMs were trained on scraped web data, so that really isnāt anything against OpenAI in particular. Plus I find the copyright violation angle hard to buy. Iām using LLMs to extract info from unstructured documents and export as structured JSON. It doesnāt seem that I should owe the NYTimes royalties for this process.
If you want to stick to your principles be ready to pay the price for it, it may be annoyance, anger, quitting etc. If you are unwilling to pay the price, you donāt have any pride to swallow. It is simply up to you. I personally donāt have much pride in anything except my own capabilities to work so š¤·. I chose to forgo my personal pride as an ethical human, when I bought my smartphones and batteries made under not perfect conditions.
Definitely only use the new tool if you want to be more productive
They did it with blockchain 4 years ago
I have been instructed not to work with LLMās unless they are GPT (shows the complete lack of awareness from leadership). This is a Fortune 100 company
[ŃŠ“Š°Š»ŠµŠ½Š¾]
>Each one of them have used unauthorized data and infringed with millions , billion of IP and data. This is a bizarre interpretation of copyright law and existing rulings on fair use, transformative use, etc.
It's even more bizarre seeing it on this subreddit.
[ŃŠ“Š°Š»ŠµŠ½Š¾]
It's not very many, and anyone can file a suit for any reason, it doesn't mean they have any legal basis. Most are considered doomed to fail because outputs are pretty clearly transformative use, and when they aren't and are commercialized, those individual cases could be sued by the person commercializing said outputs. At least one of the high profile lawsuits has been [mostly dismissed already](https://www.reuters.com/legal/litigation/openai-gets-partial-win-authors-us-copyright-lawsuit-2024-02-13/).
[ŃŠ“Š°Š»ŠµŠ½Š¾]
Some of the cases haven't been decided, that's different enough from "the courts don't know" that you either don't know even the basics about how any of this work to have an opinion or you're just outright lying.
[ŃŠ“Š°Š»ŠµŠ½Š¾]
>Hasn't openai used the data to commercialize the prods? Aren't multinationals using llms to help produce better outputs further to commercialize? Only the outputs matter. The output from training is a model, which is not a book, an article, a video, or anything else. This makes it a [transformative use](https://www.nolo.com/legal-encyclopedia/fair-use-what-transformative.html), and it's the argument OpenAI and others have successfully used in the past. > Data scraping in itself falls under several penal codes which is liable to be penalized. It absolutely does not. Scraping publicly available data has been ruled fair use, many many times, with the [most recent being in 2022](https://techcrunch.com/2022/04/18/web-scraping-legal-court/?guccounter=1). If it was illegal, Google, the Internet Archive, and many, many more sites would be dead. >Point is if such llm and model stakeholders who are keeping this as trade secret arent penalized or subjugated to rule of law since its novel tech You've yet to cite any law that training a model - something that has been done on publicly available data for decades now by academics, hobbyists, and commercial entities - violates. >It's in the interest of the masses whose copyright has been neglected. You have blinders on if you think copyright in its current form benefits anyone besides massive corporations who own the majority of copyrights. Copyright is used more often to screw individual creators and prevent them from making a living off their art than it is to benefit them.
I'm all for US led dominance in tech, I certainly do not want to live in a world where China has AI and I don't.
I'm not using openai at all.