[jewel]

We have a product that uses ChatGPT via the API, using the 3.5 turbo version. Our query involves some dates. Instead of giving back text like it usually does, today it has been giving errors because it does not think 2024-02-29 is a valid date.

This is easy to reproduce with the web interface, at least sometimes [0]. It start out by saying it's not a valid date and then as it's explaining why it isn't it realizes its mistake and sometimes corrects itself.

[0] https://chat.openai.com/share/37490c9f-81d6-499f-b491-116536...

[esalman]

Blows my mind that people consider using ChatGPT for serious applications. I mean it's fine as a code autocorrect/autocomplete tool as in GitHub copilot. But it should not replace the code itself. You encounter a bug in the code, you fix it, you never encounter it again. But ChatGPT will repeat the same mistake sooner or later. That's not how we should engineer solution for critical problems.

[Draiken]

As long as you add the "AI" keyword to the product/app/company and sell to people that don't understand how unreliable it is, you're good.

Let me rephrase that: you can profit from it. Even if it's not good.

[makoto12]

If you sandbox your connection to openAI correctly, then you can get the benefit of a llm without making your application look silly at the same time. Identifying the correct places in your business to use it is tricky, but imo it certainly makes sense in a lot of specific areas. Just not a catch all that can run your business for you

[ativzzz]

It's great for prototyping and creating outlines/rough drafts or for creating rough summaries - you can build this into some features to help your customers speed up writing lots of text

[lucumo]

If the cost-savings is worth it compared to the problems...

I mean, that's how we do it with humans. It's quite a common occurrence to keep a part of a business process human because automating it would we too expense due to edge cases.

Humans make mistakes and are expensive, but are also flexible and usually smartish. ChatGPT makes mistakes and is usually dumbish, but is also flexible and cheap.

Engineering is about picking the right trade-offs in your solution.

[littlestymaar]

Wired: LLM are practically AGI

Tired: ChatGPT thinks February 29th isn't a valid date.

[pquki4]

My favorite prompt: asking "How many e's are there in the word egregious". Always says three (ChatGPT 3.5, 4, 4 turbo). When you ask it which three, it realizes its mistake and apologizes (or sometimes tells you where they are that is completely wrong). Looks like it just outputs gibberish for these things.

[AlienRobot]

That's because it's trained on 2021 data. No February 29 back then!

[pants2]

For the record, both ChatGPT-4 and Gemini Ultra affirmed that it's a valid date. Gemini reasoned through it and GPT-4 ran python code to make sure 2024 was divisible by 4.

[peteradio]

But akshually neither does my uncle Ned

[lynx23]

Saw the same via the API with gpt-4-0125-preview. IOW, even gpt-4 thinks 2024 is not a leap year.

[fragmede]

That's weird, I don't get that with gpt-4

https://chat.openai.com/share/336b1c4b-53b7-4d56-ac68-e3c868...

[baron816]

ChatGPT thinks today is March 1. Go ahead and ask it what today’s date is.

[huytersd]

> No, 2024 is not a leap year. Leap years are years divisible by 4, except for years that are divisible by 100 unless they are also divisible by 400. Therefore, the next leap year will be 2024.

[s9df898r32h]

thats better then Mistral... it "thinks" it is March 15th 2023

"The current date is March 15, 2023. However, please note that as a large language model, my knowledge is based on the data I was trained on, which is up to 2021. Therefore, I cannot provide real-time information or updates on current events or dates. I recommend checking a reliable source such as a calendar or a trusted news website for the most accurate and up-to-date information."

[vdfs]

It is March 1 in many places

[cheapgeek]

It's UTC. Ask it what day was yesterday.

[js2]

As of 10 PM US/Eastern, ChatGPT 3.5 answers as follows for me:

> You: what is today's date?

> ChatGPT: Today's date is February 29, 2024. Please note that February 29 occurs only in leap years. If you have any more questions or if there's anything else I can help you with, feel free to ask!

[floydianspiral]

openai actually in their platform for billing, etc shows that its actually march 1 as well.