Stories with Tag Wikipedia

• Wikipedia: Database Download

  permalink

  Posted: 2025-04-27 13:21:23

  Verbose tl;dr

  Wikipedia offers free downloads of its database in various formats. These include compressed XML dumps of all content (articles, media, metadata, etc.), current and historical versions, and smaller, more specialized extracts like article text only or specific language editions. Users can also access the data through alternative interfaces like the Wikipedia API or third-party tools. The download page provides detailed instructions and links to resources for working with the large datasets, along with warnings about server load and responsible usage.

  The Wikipedia article titled "Wikipedia: Database Download" provides comprehensive information on acquiring copies of the extensive Wikipedia database. It elucidates the various methods available for obtaining this data, ranging from smaller, more manageable snapshots and topical subsets to the complete, multi-terabyte dataset. The article emphasizes that the full database is substantial and requires significant storage capacity and processing power, advising users to consider their resources carefully before attempting a download.

  The article meticulously details several download options. These include compressed XML dumps, which are updated regularly and contain the entirety of Wikipedia's content, including article text, history, metadata, and multimedia links. It also explains the availability of specific data extracts like article text only or recent changes. Furthermore, it guides users towards specialized databases like the Kiwix offline reader database, designed for portable, offline access to Wikipedia content, and the Wikidata database, a structured knowledge base separate from but linked to Wikipedia.

  The article also explores alternative access methods to Wikipedia's data beyond direct downloads. These include accessing the database replicas, utilizing the Wikipedia API, and querying structured data through Wikidata Query Service. These methods are particularly useful for specific data retrieval or analysis, avoiding the need to download and process the entire dataset. The article offers links and detailed instructions for each access method.

  The "Wikipedia: Database Download" article goes beyond mere download instructions by offering guidance on the technical aspects of handling the downloaded data. It discusses the formats used, such as XML and SQL, and recommends tools and software for processing and parsing the data. Furthermore, it acknowledges the potential challenges related to the sheer volume of data and offers practical tips for efficient processing. The page also mentions the licensing of the data under the Creative Commons Attribution-ShareAlike license and provides information about database dumps policy regarding redistribution and mirroring. Finally, it maintains a section for external links that provide access to tools and services that can assist users in working with the Wikipedia database. This makes it a valuable resource for anyone seeking to utilize Wikipedia's vast repository of knowledge for research, development, or offline access.

  • Wikipedia
  • Database
  • Download
  • Data Dump
  • offline access
  • Open Data
  • Wikimedia
  • Wiki Data
  • information retrieval
  • SQL
  • XML
  • Data Analysis
  • Research
  • Archive
  • backup

  Summary of Comments ( 39 )
  https://news.ycombinator.com/item?id=43811732

  Verbose tl;dr

  Hacker News users discussed various aspects of downloading and using Wikipedia's database. Several commenters highlighted the resource intensity of processing the full database, with mentions of multi-terabyte storage requirements and the need for significant processing power. Some suggested alternative approaches for specific use cases, such as using Wikipedia's API or pre-processed datasets like the one offered by the Wikimedia Foundation. Others discussed the challenges of keeping a local copy updated and the potential legal implications of redistributing the data. The value of having a local copy for offline access and research was also acknowledged. There was some discussion around specific tools and formats for working with the downloaded data, including tips for parsing and querying the XML dumps.

  The Hacker News post titled "Wikipedia: Database Download" (https://news.ycombinator.com/item?id=43811732) has a moderate number of comments discussing various aspects of downloading and using Wikipedia's database dumps.

  Several comments focus on the practical challenges and considerations related to downloading and processing the large datasets. One user points out the significant disk space requirements, even for compressed versions of the dumps, advising potential downloaders to carefully assess their storage capacity. Another comment highlights the computational resources needed to process the data, mentioning the RAM and processing power required for tasks like parsing and indexing. A separate thread discusses the various download options, including using BitTorrent for faster downloads and the availability of smaller, more specific dumps for those not needing the entire dataset.

  Some users discuss the utility of having a local copy of Wikipedia. One comment mentions using the Kiwix offline reader, which allows access to a local copy of Wikipedia without the need for complex processing. Others discuss the potential for using the data for research, natural language processing tasks, and personal projects like building a local search engine. A particular comment thread delves into the technical details of setting up a local search index using tools like Xapian and Lucene.

  The licensing of the Wikipedia data is also a topic of discussion. A user clarifies that the data is available under the Creative Commons license, emphasizing the importance of proper attribution when using the content.

  A few comments touch on the history of Wikipedia dumps and how the process has evolved over time. One user reminisces about downloading Wikipedia dumps on DVDs in the past.

  While there isn't a single overwhelmingly compelling comment, the discussion as a whole provides valuable insights into the practicalities and potential uses of the Wikipedia database dumps, covering aspects like hardware requirements, software tools, licensing, and the historical context of data availability. The collective knowledge shared by the commenters offers a comprehensive guide for anyone considering working with Wikipedia's data offline.

• Wikipedia’s nonprofit status questioned by D.C. U.S. attorney

  permalink

  Posted: 2025-04-25 23:03:55

  Verbose tl;dr

  The U.S. attorney for the District of Columbia, Matthew Graves, has questioned Wikimedia Foundation's nonprofit status. In a letter to the foundation, Graves raised concerns about potential misuse of donations, citing large reserves, high executive compensation, and expenditures on projects seemingly unrelated to its core mission of freely accessible knowledge. He suggested these activities could indicate private inurement or private benefit, violations that could jeopardize the foundation's tax-exempt status. The letter requests information regarding the foundation's finances and governance, giving a deadline for response. While Wikimedia maintains confidence in its compliance, the inquiry represents a significant challenge to its operational model.

  In a development that has sent ripples through the philanthropic and digital information landscapes, the United States Attorney for the District of Columbia, Matthew M. Graves, has penned a letter to the Wikimedia Foundation, the non-profit organization that operates the ubiquitously consulted online encyclopedia, Wikipedia. This missive, dated April 18, 2025, and subsequently brought to light by the Washington Post’s coverage on April 25th of the same year, raises significant questions regarding the Wikimedia Foundation’s continued adherence to the tenets and regulations governing its 501(c)(3) non-profit status under the United States Internal Revenue Code.

  Specifically, Mr. Graves articulates concerns surrounding the Foundation’s substantial and ever-growing endowment, coupled with what he perceives as a disparity between the accumulation of these funds and the comparatively modest disbursement of grants and expenditures directed towards the actual advancement of the Foundation’s stated mission: the provision of free and accessible knowledge to the global populace. The letter characterizes the Foundation’s financial practices as potentially indicative of an organization prioritizing asset accumulation over active deployment of resources in the pursuit of its publicly declared charitable objectives.

  Furthermore, the U.S. Attorney’s communication delves into the specifics of Wikipedia’s operational structure, raising concerns regarding the allocation of financial resources towards what is described as extensive internal infrastructure, including a sizeable staff and a sophisticated technological framework. Mr. Graves posits that these expenditures, while undeniably contributing to the platform’s continued functionality, may not align perfectly with the expectations placed upon a tax-exempt charitable entity, implicitly suggesting a possible diversion of resources away from direct charitable activities.

  The letter explicitly refrains from accusing the Wikimedia Foundation of any wrongdoing or legal violation. However, it unequivocally requests a comprehensive accounting of the Foundation’s financial activities, demanding detailed explanations regarding its spending practices and the rationale behind its significant endowment growth. This request signals a potentially protracted period of scrutiny for the Wikimedia Foundation, as it navigates the complexities of justifying its financial model within the stringent regulatory framework governing non-profit organizations in the United States. The implications of this inquiry could be far-reaching, potentially impacting the future operational model and financial strategies employed by the Foundation, and consequently influencing the continued accessibility and development of one of the internet’s most vital and widely utilized resources.

  • Wikipedia
  • nonprofit
  • Wikimedia Foundation
  • IRS
  • Tax Exempt Status
  • US Attorney
  • DC
  • Investigation
  • Donation
  • Fundraising
  • transparency
  • legal
  • Technology
  • Internet
  • Online Encyclopedia

  Summary of Comments ( 682 )
  https://news.ycombinator.com/item?id=43799302

  Verbose tl;dr

  Several Hacker News commenters express skepticism about the US Attorney's investigation into Wikimedia's non-profit status, viewing it as politically motivated and based on a misunderstanding of how Wikipedia operates. Some highlight the absurdity of the claims, pointing out the vast difference in resources between Wikimedia and for-profit platforms like Google and Facebook. Others question the letter's focus on advertising, arguing that the fundraising banners are non-intrusive and essential for maintaining a free and open encyclopedia. A few commenters suggest that the investigation could be a pretext for more government control over online information. There's also discussion about the potential impact on Wikimedia's fundraising efforts and the broader implications for online non-profits. Some users point out the irony of the US government potentially hindering a valuable resource it frequently utilizes.

  The Hacker News post titled "Wikipedia’s nonprofit status questioned by D.C. U.S. attorney" (linking to a Washington Post article) generated a moderate amount of discussion, with several commenters raising pertinent points about the nature of Wikipedia's operations and the implications of the U.S. attorney's inquiry.

  Several commenters focused on the apparent contradiction between Wikipedia's non-profit status and its substantial fundraising efforts. They questioned whether the aggressive fundraising tactics, coupled with the significant reserves accumulated by the Wikimedia Foundation, align with the spirit of a non-profit organization. Some suggested that Wikipedia's fundraising appeals create a perception of financial need that doesn't seem justified given their existing funds. This led to discussions about the appropriate level of reserves for a non-profit and the ethics of continually soliciting donations.

  Another key theme in the comments was the perceived shift in Wikipedia's content and editorial policies. Some users expressed concerns about increasing bias and a decline in neutrality, potentially influenced by the interests of major donors. The discussion touched on the challenges of maintaining objectivity in a crowdsourced platform and the potential impact of external pressures on editorial decisions.

  Several commenters also discussed the implications of the U.S. attorney's investigation, with some speculating about the potential outcomes and the legal basis for challenging Wikipedia's non-profit status. There was also debate about the role of government oversight in the non-profit sector and the potential for this case to set a precedent for other organizations.

  A few commenters defended Wikipedia, arguing that its fundraising practices are necessary to maintain and expand its operations, and highlighting the value of its free and accessible information. They pointed out the complexities of running a global platform and the need for substantial resources to combat vandalism, misinformation, and other challenges.

  While there wasn't overwhelming consensus on any particular point, the comments collectively reflected a concern about the transparency and accountability of Wikipedia's operations, particularly regarding its finances and editorial policies. The discussion highlighted the tension between maintaining a non-profit status while operating on a large scale and soliciting substantial donations.

• Photographers Are on Mission to Fix Wikipedia's Famously Bad Celebrity Portraits

  permalink

  Posted: 2025-03-11 03:15:59

  Verbose tl;dr

  Professional photographers are contributing high-quality portraits to Wikipedia to replace the often unflattering or poorly lit images currently used for many celebrity entries. Driven by a desire to improve the visual quality of the encyclopedia and provide a more accurate representation of these public figures, these photographers are donating their work or releasing it under free licenses. They aim to create a more respectful and professional image for Wikipedia while offering a readily available resource for media outlets and the public.

  Within the expansive digital encyclopedia that is Wikipedia, a collaborative effort of global proportions, a peculiar incongruity has been observed and is now being addressed: the often-subpar quality of photographic portraits representing notable individuals, particularly those of celebrity status. These images, intended to provide a visual accompaniment to biographical information, frequently fall short of the standards expected for such prominent figures, suffering from issues such as poor lighting, unflattering angles, low resolution, or simply a lack of aesthetic appeal. This phenomenon, long a source of mild amusement and occasional consternation for Wikipedia users, has prompted a dedicated cadre of professional and amateur photographers to embark on a mission of photographic rectification.

  These individuals, driven by a desire for accuracy, completeness, and visual enhancement of the online encyclopedia, are actively contributing their photographic skills and resources to replace these inadequate depictions with higher-quality alternatives. Their endeavors involve meticulous research, often extending beyond the readily available image repositories, to locate and obtain superior photographic representations. This process may involve contacting photographers directly, seeking permission for image use, or diligently exploring licensed image databases. The motivation behind this undertaking is multifaceted. It stems from a recognition that Wikipedia serves as a primary source of information for a vast global audience, and the quality of its visual content should therefore reflect the significance of the individuals it represents. Moreover, it acknowledges the power of a well-crafted portrait to convey personality, capture essence, and enhance the overall understanding of a subject.

  The initiative is not without its challenges. Navigating copyright restrictions, obtaining appropriate permissions, and ensuring the accuracy of the chosen images require careful attention and often a significant investment of time. Furthermore, achieving consensus within the Wikipedia community regarding the suitability of replacement images can be a complex process. Despite these hurdles, the photographers involved demonstrate a sustained commitment to elevating the visual standard of Wikipedia's biographical entries, one portrait at a time. Their collective efforts contribute not only to an improved aesthetic experience for users but also to a more dignified and respectful representation of notable figures within the digital realm. This ongoing project serves as a testament to the power of collaborative effort and the dedication of individuals to enhance the quality and accuracy of freely accessible information.

  • Wikipedia
  • photography
  • Portraits
  • Celebrities
  • Image Quality
  • Online Encyclopedia
  • Wikimedia Commons
  • Open Source
  • Community Editing
  • Visual Media

  Summary of Comments ( 27 )
  https://news.ycombinator.com/item?id=43328835

  Verbose tl;dr

  HN commenters generally agree that Wikipedia's celebrity photos are often unflattering or outdated. Several suggest that the issue isn't solely the photographers' fault, pointing to Wikipedia's stringent image licensing requirements and complex upload process as significant deterrents for professional photographers contributing high-quality work. Some commenters discuss the inherent challenges of representing public figures, balancing the desire for flattering images with the need for neutral and accurate representation. Others debate the definition of "bad" photos, arguing that some unflattering images simply reflect reality. A few commenters highlight the role of automated tools and bots in perpetuating the problem by automatically selecting images based on arbitrary criteria. Finally, some users share personal anecdotes about attempting to upload better photos to Wikipedia, only to be met with bureaucratic hurdles.

  The Hacker News post "Photographers Are on Mission to Fix Wikipedia's Famously Bad Celebrity Portraits" has generated several comments discussing the initiative to improve Wikipedia's often unflattering or low-quality celebrity photos.

  Several commenters express appreciation for the effort, pointing out how poor quality images can detract from the overall perception of Wikipedia as a reliable source. They note that blurry, poorly lit, or awkwardly cropped photos can make even well-known individuals appear less credible or even comical. The initiative to replace these with professional, high-resolution images is seen as a positive step towards enhancing the platform's visual appeal and professionalism.

  Some comments focus on the technical aspects of photography and image licensing. One commenter highlights the importance of understanding how different cameras and lenses can impact the final image, suggesting that even a seemingly simple portrait can be influenced by factors like focal length and compression. Others discuss the complexities of Creative Commons licensing and the importance of ensuring that images uploaded to Wikipedia comply with the platform's guidelines.

  A few comments touch on the subjective nature of portraiture. While acknowledging the need for high-quality images, they also point out that what constitutes a "good" portrait can be open to interpretation. They suggest that factors like lighting, posing, and even the photographer's personal style can influence how a subject is perceived, making it difficult to establish a universal standard for quality.

  One commenter humorously observes the phenomenon of Wikipedia editors reverting seemingly improved images back to older, less flattering versions, sometimes due to stringent interpretations of copyright rules or personal preferences. This anecdote sparked a brief discussion about the challenges of maintaining consistency and quality control on a collaboratively edited platform like Wikipedia.

  The discussion also extends to the broader issue of image quality on the internet, with some commenters expressing frustration at the prevalence of low-resolution or compressed images on various websites. The effort to improve Wikipedia's celebrity portraits is seen as a potential model for other platforms to follow.

  Finally, some comments mention the potential ethical considerations of replacing existing images, even if they are of poor quality. They raise questions about the potential for bias in selecting replacement images and the importance of maintaining a neutral point of view.