• 3 Posts
  • 86 Comments
Joined 1 year ago
cake
Cake day: August 19th, 2023

help-circle
  • canihasaccount@lemmy.worldtoScience Memes@mander.xyzShe-Ra Lives!
    link
    fedilink
    English
    arrow-up
    17
    arrow-down
    1
    ·
    3 days ago

    A bit of an exaggeration, sure. But only a bit. The lay summary of the article I referenced states the following:

    Venkataraman et al. find that the paper commits every error that it was possible to make in the paper: leaving out important papers, including irrelevant papers, using duplicate papers, mis-coding their societies, getting the wrong values for “big” versus “small” game, and many others.

    “commits every error that it was possible to make in the paper,” and, “completely incorrect,” aren’t very different.


















  • Claude Opus disagrees, lol (edit to add: all of what follows is Claude; not me):

    I respectfully disagree with the claim that current language models are logically incapable of mapping language to actual concepts. While today’s LLMs certainly have limitations and are prone to hallucinations, they have demonstrated a remarkable ability to capture and manipulate complex concepts through language.

    A few key points:

    LLMs are trained on vast amounts of human-generated text which is grounded in real-world concepts, allowing the models to build rich representations that go beyond surface-level word associations. Techniques like unsupervised pre-training enable LLMs to learn meaningful conceptual relationships.
    
    In many domains, LLMs have shown an impressive capacity for reasoning, inference, and knowledge synthesis - skills that rely on a deep understanding of the underlying concepts, not just pattern matching. Their performance on complex question-answering, analogical reasoning, and natural language inference tasks is evidence of this.
    
    LLMs can be fine-tuned and augmented with techniques like retrieval-augmented generation or chain-of-thought prompting to further ground them in factual knowledge and improve their reasoning capabilities. This allows more robust conceptual mappings.
    
    Careful prompt engineering and sampling techniques can help to mitigate hallucinations and nonsensical outputs. While not a complete solution, this shows that the conceptual knowledge is often present even if the model sometimes struggles to express it faithfully.
    

    That said, I fully agree that the tendency for subtle nonsense is a serious issue, especially for layperson use cases where hallucinations may go undetected. Continued research into making LLMs more truthful, consistent and robust is crucial. Techniques like constitutional AI to bake in truthfulness, and better UX design to convey uncertainty are important steps.

    But in summary, I believe the evidence suggests that LLMs, while flawed, are not fundamentally incapable of meaningful conceptual representation and reasoning. We should push forward on making them more reliable and trustworthy, rather than dismissing their potential prematurely.