The blog post explores the potential of generative AI in historical research, showcasing its utility through three case studies. The author demonstrates how ChatGPT, Claude, and Bing AI can be used to summarize lengthy texts, analyze historical events from multiple perspectives, and generate creative content such as fictional dialogues between historical figures. While acknowledging the limitations and inaccuracies these models sometimes exhibit, the author emphasizes their value as tools for accelerating research, brainstorming new interpretations, and engaging with historical material in novel ways, ultimately arguing that they can augment, rather than replace, the work of historians.
The Substack post "The leading AI models are now very good at making stuff up about history" by Res Obscura explores the burgeoning intersection of generative artificial intelligence and historical research, specifically examining how these powerful new tools can be utilized – and misused – within the field. The author meticulously details three distinct case studies to illustrate both the potential benefits and significant pitfalls of incorporating AI language models into the historian's workflow.
The first case study focuses on using generative AI for idea generation and exploratory research. The author tasked an AI model with developing potential research questions surrounding a relatively obscure historical topic: the history of pencil sharpeners. While acknowledging the model's propensity for fabrication, the author highlights its capacity to stimulate new avenues of inquiry and uncover previously unconsidered perspectives by swiftly generating a multitude of questions, some insightful and others nonsensical. This rapid ideation process, the author argues, can be valuable in the early stages of research, offering a springboard for further investigation and helping historians break free from pre-conceived notions.
The second case study delves into the use of AI for source summarization, specifically focusing on digests of primary source texts. The author demonstrates how AI can condense lengthy historical documents, potentially saving researchers considerable time and effort in the initial stages of source analysis. However, the post emphasizes the critical importance of meticulous fact-checking. The author reveals how the AI, while capable of producing seemingly coherent summaries, often introduces subtle inaccuracies and outright fabrications, highlighting the inherent danger of relying solely on AI-generated interpretations without rigorous verification against the original source material.
The third and final case study investigates the application of AI for translation, particularly with archaic or less common languages. The author illustrates how AI can offer provisional translations of historical texts, providing researchers with a preliminary understanding of the material even in the absence of specialized linguistic expertise. Yet again, the author underscores the necessity of caution and corroboration. The AI's translations, while sometimes impressively accurate, are also prone to errors, particularly in nuances of meaning and cultural context. The post stresses that AI-generated translations should be treated as a starting point, requiring careful scrutiny and comparison with expert translations or further linguistic analysis whenever possible.
Ultimately, the post concludes that generative AI, while presenting exciting new possibilities for historical research, should be employed judiciously and with a keen awareness of its limitations. The author advocates for a symbiotic relationship between historian and AI, wherein the technology serves as a powerful assistant, augmenting but not replacing the researcher's critical thinking, rigorous methodology, and deep contextual understanding. The post emphasizes the vital importance of skepticism, verification, and the continued primacy of established historical research practices in the face of these rapidly evolving technological advancements.
Summary of Comments ( 190 )
https://news.ycombinator.com/item?id=42798649
HN users discussed the potential benefits and drawbacks of using generative AI for historical research. Some expressed enthusiasm for its ability to quickly summarize large bodies of text, translate languages, and generate research ideas. Others were more cautious, highlighting the potential for hallucinations and biases in the AI outputs, emphasizing the crucial need for careful fact-checking and verification. Several commenters noted that these tools could be most useful for exploratory research and generating hypotheses, but shouldn't replace traditional methods. One compelling comment suggested that AI might be especially helpful for "distant reading" approaches to history, allowing for the analysis of large-scale patterns and trends in historical texts. Another interesting point raised the possibility of using AI to identify and analyze subtle biases present in historical sources. The overall sentiment was one of cautious optimism, acknowledging the potential power of AI while recognizing the importance of maintaining rigorous scholarly standards.
The Hacker News post titled "Using generative AI as part of historical research: three case studies" linking to an article on Res Obscura has generated a few comments, mostly focusing on the limitations and potential pitfalls of using AI in historical research, rather than outright enthusiasm.
One commenter expresses skepticism about the practicality of using AI for this purpose, pointing out that while AI might be able to generate plausible-sounding narratives, it lacks the ability to critically evaluate sources and distinguish between reliable and unreliable information, a crucial skill for any historian. They argue that the real work of historical research lies in the meticulous examination of primary sources and the careful construction of arguments based on evidence, something AI cannot currently replicate. This commenter essentially sees AI as more of a novelty than a genuinely useful tool for historians.
Another commenter echoes this sentiment, suggesting that the current capabilities of AI are more suited to tasks like summarizing existing historical narratives rather than generating new historical insights. They also emphasize the importance of understanding the biases inherent in AI models, which are trained on existing data and therefore prone to perpetuating existing historical narratives and potentially overlooking marginalized perspectives. This commenter also cautions against the potential for AI to fabricate information, creating seemingly plausible but ultimately false historical accounts.
A third commenter raises the issue of copyright and intellectual property, questioning whether text generated by AI based on copyrighted historical sources could be considered a derivative work and therefore subject to copyright restrictions. They highlight the legal ambiguities surrounding AI-generated content and the potential for future legal challenges.
One commenter offers a slightly more optimistic perspective, suggesting that AI could be useful for generating initial drafts or summaries, which historians could then refine and verify. However, even this commenter acknowledges the limitations of AI and emphasizes the need for human oversight and critical evaluation.
In summary, the comments on the Hacker News post express a cautious and somewhat skeptical view of the potential of AI in historical research. While some see limited potential for AI to assist with certain tasks, the overall sentiment is that AI lacks the critical thinking skills, source evaluation abilities, and nuanced understanding of context that are essential for serious historical scholarship. Furthermore, commenters highlight the potential for AI to perpetuate biases, fabricate information, and raise copyright concerns.