Wiki[mp]edia data sources & the MediaWiki API

•

2 recomendaciones•1,818 vistas

Brianna Laugher

For #melhack - http://lplabs.com/melbournehack/pmwiki/pmwiki.php/Main/HomePage

Tecnología

Wiki[mp]edia
data sources &
the MediaWiki API
Brianna Laugher
for #melhack
November 2009

Wikipedia
13M articles total
3M+ articles in English
240+ languages
Simple English!

${{coord|37|48|49|S|144|57| 47|E|type:city_region:AU-VIC| display=inline,title}} stable.toolserver.org/geohack/ wiki.toolserver.org/view/GeoHack$

${{Infobox Company |name = Lonely Planet |logo = |type = [[United Kingdom|British]] [[Government-owned company|government-owned]] (subsidiary of [[BBC Worldwide]]) |genre = [[Guide book|Travel guides]] |foundation = 1972 |founder = Tony Wheeler Maureen Wheeler |location_city = [[Footscray, Victoria]] |location_country = [[Australia]] |location = |origins = |key_people = Matt Goldberg (Global [[CEO]]) |area_served = Worldwide |industry = [[Multi media]] |products = Travel [[guidebook, digital applications, online travel community]] |services =$

Wikimedia Commons
commons.wikimedia.org
Multilingual
5M+ files
“Self-created”, PD, Flickr

Predominantly photographs,
but also diagrams, maps, flags

Wiktionary
5M+ entries
170+ languages
13 languages > 100K entries

French biggest at 1.5M
(English second at 1.4M)

JavaScript Wiktionary lookup plugin for
third parties:
http://bawolff.blogspot.com/2009/10/introducing-
wiktionary-lookup-now-for.html

http://en.wiktionary.org/wiki/Wiktionary:Parsing

MediaWiki structure
 Users
 Logs

 Pages, subpages, talk pages

 Links, backlinks

 Templates

 Categories

MediaWiki markup

The only thing that completely
understands it is MediaWiki :(

Database dumps
XML

download.wikimedia.org
OR Amazon Public Data Sets

meta.wikimedia.org/wiki/
Data_dumps

DBpedia
Community project extracting
structured data from Wikipedia and
making it available

Can download data sets or query them
online

Ontology++

e.g. dbpedia.org/page/Lonely_Planet

MediaWiki API

mediawiki.org/wiki/API

en.wikipedia.org/w/api.php

Client libraries!

mwclient

Python library for
accessing MediaWiki
APIs

Toolserver
toolserver.org

Server for community-developed plugins,
addons, extensions, stats and hacks –
tools

Tools often explicitly implements implicit
editing community standards
(“community API”)

TemplateTiger
toolserver.org/~kolossos/templatetiger/

For a few dozen Wikipedia languages, &
Wikimedia Commons

Lets you query templates very much
like SQL

Thanks!
identi.ca/pfctdayelise
blaugher@wikimedia.org.au

Logos and screenshots
may be copyright
their respective owners
Slides are otherwise
© Brianna Laugher

Más contenido relacionado

Similar a Wiki[mp]edia data sources & the MediaWiki API

Web2 0

rbwalker15

Web20 Intro Naj Shaik

Karen Vignare

Hacking Mediawiki

Jani Patokallio

Aswc2009 Smw Tutorial Part 1 Intro And Examples

Jesse Wang

Web2.0 Tools

Satyendra Arora

Reflections On Personal Experiences In Using Wikis

lisbk

Web 2.0 Resources for Teaching

Alexis Alexander

The Social Semantic Web

John Breslin

Web 2.0: Implications For The Cultural Heritage Sector

lisbk

Water Wiki, a global platform for water community

XWiki

Web 2.0 lib_2.0_1

smtcd

Semantic Wikis - Social Semantic Web in Action

wikis

Wikis[1]

wikis

wikis

Course Wikis

Alternative eLearning Tools

Yum Studio

The “right to fork”, a consequence of the “hack on copyright” that is copyleft licensing, helps keep open source and open content project leaders honest. Forking is a political act as much as a version control command, and it used to be that both were a big deal. But now that distributed version control systems (DVCS) have made forking trivial, are there implications for the political act as well? How does political forking work within collaborative prose text projects (i.e. wikis)? English Wikipedia is so large as to be practically unforkable - it essentially has an unassailable monopoly, and unchecked power, in the English language encyclopedia market. One of the core Wikipedia rules is “one topic, one article”, which would seem to prohibit forking, but could we adhere to this principle and still take advantage of DVCS? Can a community be forked while keeping the shared project goals intact? Audience members will benefit from a grasp of version control, distributed version control and the workings of wikis and Wikipedia. Presented at the 'Freedom in the Cloud' miniconf, Monday January 24 2011 at linux.conf.au.

Distributed wikis

Brianna Laugher

New Free Web 2.0 Developments 97

gueste036e7

Similar a Wiki[mp]edia data sources & the MediaWiki API (20)

Web2 0

Web20 Intro Naj Shaik

Hacking Mediawiki

Aswc2009 Smw Tutorial Part 1 Intro And Examples

Web2.0 Tools

Reflections On Personal Experiences In Using Wikis

Web 2.0 Resources for Teaching

The Social Semantic Web

Web 2.0: Implications For The Cultural Heritage Sector

Water Wiki, a global platform for water community

Web 2.0 lib_2.0_1

Semantic Wikis - Social Semantic Web in Action

wikis

Wikis[1]

wikis

Course Wikis

Alternative eLearning Tools

Distributed wikis

New Free Web 2.0 Developments 97

Más de Brianna Laugher

Talk presented at PyConline Australia 2021. So you’re a software developer, now what? Is it satisfying? How do you level up? How can you influence your career direction? How can you even figure out what you want? If your manager asked where you want your career to go, do you know what you would say? Do you feel like being a developer is a bit “meh”, but you are not really sure what else is out there? If career questions make you feel a bit lost, you’re not alone. It’s common for software developers to feel that their career is something that happens to them while they scramble for job stability, rather than something that they have any influence or control over. This talk will challenge you to look sideways when it comes to moving on up. Software is so much more than just programming, even if you still program every day. Come to get some new ideas on what your current or next job might look like, and how to figure out what you might like. Leave with some confident directions to guide your next 1:1 discussion or job search. Software is a generous and expansive field - why not aim to find fulfilling work while we’re here, hey?

So You're A Software Developer, Now What? Exploring Career Growth

Brianna Laugher

CFFSW - Crowdfunded free software

Brianna Laugher

Slides for a talk at PyCon AU 2013. Integrating PyDAP + WMS + OpenLayers + IPython Notebook. Video: http://www.youtube.com/watch?v=YJqBGi48RAM The IPython Notebook is a powerful web app for exploring ideas and data sets with Python. It has excellent integration with Matplotlib, giving the user highly customisable static plots with ease. But for larger data sets, a static plot may not be ideal - the ability to pan, zoom, choose dynamic layers and sample the data at particular points would be nice. This talk will demonstrate just how easy it is to integrate a Web Map Service/client such as Pydap/Leaflet.js into the IPython Notebook.

Dynamic viz in the IPython Notebook

Brianna Laugher

Video - http://www.youtube.com/watch?v=DTNejE9EraI Code - https://gist.github.com/3386951 Pytest is a mature and comprehensive testing suite for Python projects, but it can be a little intimidating for newcomers. Where do these mysterious funcargs come from, how do parametrised tests work, and where are my xUnit-style setUp and tearDown methods? Pytest lives by "convention over configuration" - which is great once you know what the conventions are. This talk will look at real examples of pytest in use, emphasising the features that differentiate it from nose.

Funcargs & other fun with pytest

Brianna Laugher

A talk presented at PyCon AU 2011. Zookeepr ( http://zookeepr.org/ ) is a comprehensive web-based conference management system, written in Python and built on Pylons. It has an unusual development history: custom created for the annual Linux.conf.au conference, there are yearly spikes in event-focused feature development, but relatively little of the ongoing development typically seen in open source projects. This presentation is an introduction to the project, aimed at developers interested in contributing to a non-trivial open source project where meeting your fellow developers is quite possible, even likely, and your work is almost guaranteed to be seen and used each year by hundreds of Australia's most diehard geeks.

Zookeepr: Home-grown conference management software

Brianna Laugher

BarCamp Geelong - Why gender should be a text field

Brianna Laugher

Neurosexism

Brianna Laugher

Talk given at the Get It Louder 'SHARISM' conference in Shanghai, 22 October 2010. http://www.getitlouder.com/enChair.aspx?ID=94 One of the benefits of the open web is that good ideas can flourish easily. In the Chinese speaking web, the idea of an online encyclopedia has been especially fruitful. With the Chinese Wikipedia enjoying its eighth birthday last month, it’s worth examining whether the fragmentation of efforts ultimately leads to a better product and bigger communities, or if the “us vs them” mentality is harmful to sharing.

Clash of the encyclopedias - is competition good for sharing?

Brianna Laugher

Language Sleuthing HOWTO with NLTK

Brianna Laugher

Visualising geo-data

Brianna Laugher

GLAM-WIKI - Wikimedia tech infrastructure

Brianna Laugher

The right level of detail (MediaWiki, APIs)

Brianna Laugher

Free as in Market: Liberty and Property - Rusty Russell

Brianna Laugher

AUTHOR: CLAUDINE CHIONH Public history is a field that seeks to return the study and interpretation of history to the wider community. While 'professional' historians with academic training may be involved, public history is largely focused on and driven by 'amateurs' – genealogists, local history societies and others who collect and collate sources and interpret and present them for a general audience. The increased use of the internet in recent years has created new opportunities for opening up sources and archives to the world, and for collaboration and public presentation of historical research. In this presentation I will describe two projects that aim to collect and publish historical sources on the web and to work with local and family historians to maintain these as dynamic historical resources.

Public history in the digital age - Claudine Chionh

Brianna Laugher

It's all fun and games until someone wants to sue you: Reporting in the age o...

Brianna Laugher

OpenAustralia - Everyday democracy for everybody in Australia - Matthew Landauer

Brianna Laugher

AUTHOR: JESSICA COATES Over the last few years the free culture movement has moved out of the bedrooms and into the boardrooms. Open copyright models are increasingly of interest not just to enthusiasts, but to big name creators, government bodies and even corporate entities. It finally feels like we're just a hair's breadth away from widespread acceptance and take up, both in Australia and internationally. But there is still that tiny gap. How do we get people to cross the final divide from interested party to adopter? Drawing on her experience as Project Manager of Creative Commons Australia, Jessica Coates will lead a group discussion on tactics for promoting greater adoption of open copyright in Australia. What tools are most useful for spreading the word? What projects have been successful in the past? How do we convince people to give more thought to their copyright decisions?

Freedom Fighting: How do we convince the powers that be to relax their grip? ...

Brianna Laugher

AUTHOR: SIMON GREENER This talk will attempt a review of the geospatial data space within Australia. The talk will outline who the main players are, what spatial data is available, and the licensing options that cover their use. An assessment of the licenses will be made. In particular the talk will outline the data that is available for free and, and after establishing the various uses of that data, assess how important that data is to various sectors and individuals within society and how it might benefit society as a whole.

Free and open geodata: From shadows to reality - Simon Greener

Brianna Laugher

AUTHOR: LAURA SIMES Australia's copyright laws are the chief means by which we as a society regulate the creation and distribution of knowledge. With the digital age well and truly upon us, individuals, corporations and Governments are grappling with how copyright law now 'fits' in with this, with diverse opinions on how copyright law could or should adapt to meet these changes. This talk will look at a few current issues of interest in the world of copyright, and consider where we seem to be heading. Some developments, such as the Anti-Counterfeiting Trade Agreement (ACTA) suggest a preoccuopation with stronger copyright laws and penalties. On the other hand, the recent Review of the National Innovation System (aka the Cutler Report) presents a number of recommendations for more 'flexible' copyright law, along with a suggestion that we need to look at copyright law in a different way than we have in the past.

Future directions for copyright law - Laura Simes

Brianna Laugher

AUTHOR:ARTHUR SALE The Open Source movement, of which Linux is a shining example, is a showcase of how accessibility makes for excellence. A parallel thrust is currently being conducted in the research institutions and the publishing industries of the world to create Open Access to the world’s publicly funded research. Arthur Sale will trace the origin of the movement, its economics and the forces holding it back, and where we are now, particularly in Australia. Open Access, or OA, has very many more active participants than Open Source, and many more nay-sayers, cautious Scrooges, and ignorant people. The struggle is titanic – the benefits equally large! http://freeasinfreedom.modernthings.org/d/doku.php?id=arthur_sale

Beyond Open Source - Arthur Sale

Brianna Laugher

Más de Brianna Laugher (20)

So You're A Software Developer, Now What? Exploring Career Growth

CFFSW - Crowdfunded free software

Dynamic viz in the IPython Notebook

Funcargs & other fun with pytest

Zookeepr: Home-grown conference management software

BarCamp Geelong - Why gender should be a text field

Neurosexism

Clash of the encyclopedias - is competition good for sharing?

Language Sleuthing HOWTO with NLTK

Visualising geo-data

GLAM-WIKI - Wikimedia tech infrastructure

The right level of detail (MediaWiki, APIs)

Free as in Market: Liberty and Property - Rusty Russell

Public history in the digital age - Claudine Chionh

It's all fun and games until someone wants to sue you: Reporting in the age o...

OpenAustralia - Everyday democracy for everybody in Australia - Matthew Landauer

Freedom Fighting: How do we convince the powers that be to relax their grip? ...

Free and open geodata: From shadows to reality - Simon Greener

Future directions for copyright law - Laura Simes

Beyond Open Source - Arthur Sale

Último

Following the popularity of "Cloud Revolution: Exploring the New Wave of Serverless Spatial Data," we're thrilled to announce this much-anticipated encore webinar. In this sequel, we'll dive deeper into the Cloud-Native realm by uncovering practical applications and FME support for these new formats, including COGs, COPC, FlatGeoBuf, GeoParquet, STAC, and ZARR. Building on the foundation laid by industry leaders Michelle Roby of Radiant Earth and Chris Holmes of Planet in the first webinar, this second part offers an in-depth look at the real-world application and behind-the-scenes dynamics of these cutting-edge formats. We will spotlight specific use-cases and workflows, showcasing their efficiency and relevance in practical scenarios. Discover the vast possibilities each format holds, highlighted through detailed discussions and demonstrations. Our expert speakers will dissect the key aspects and provide critical takeaways for effective use, ensuring attendees leave with a thorough understanding of how to apply these formats in their own projects. Elevate your understanding of how FME supports these cutting-edge technologies, enhancing your ability to manage, share, and analyze spatial data. Whether you're building on knowledge from our initial session or are new to the serverless spatial data landscape, this webinar is your gateway to mastering cloud-native formats in your workflows.

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Safe Software

🐬 The future of MySQL is Postgres 🐘

RTylerCroy

Discord is a free app offering voice, video, and text chat functionalities, primarily catering to the gaming community. It serves as a hub for users to create and join servers tailored to their interests. Discord’s ecosystem comprises servers, each functioning as a distinct online community with its own channels dedicated to specific topics or activities. Users can engage in text-based discussions, voice calls, or video chats within these channels. Understanding Discord Servers Discord servers are virtual spaces where users congregate to interact, share content, and build communities. Servers may revolve around gaming, hobbies, interests, or fandoms, providing a platform for like-minded individuals to connect. Communication Features Discord offers a range of communication tools, including text channels for messaging, voice channels for real-time audio conversations, and video channels for face-to-face interactions. These features facilitate seamless communication and collaboration. What Does NSFW Mean? The acronym NSFW stands for “Not Safe For Work,” indicating content that may be inappropriate for professional or public settings. NSFW Content NSFW content encompasses material that is sexually explicit, violent, or otherwise graphic in nature. It often includes nudity, profanity, or depictions of sensitive topics.

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

UK Journal

GenAI Risks & Security Meetup 01052024.pdf

lior mazor

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Juan lago vázquez

Webinar Recording: https://www.panagenda.com/webinars/why-teams-call-analytics-is-critical-to-your-entire-business Nothing is as frustrating and noticeable as being in an important call and being unable to see or hear the other person. Not surprising then, that issues with Teams calls are among the most common problems users call their helpdesk for. Having in depth insight into everything relevant going on at the user’s device, local network, ISP and Microsoft itself during the call is crucial for good Microsoft Teams Call quality support. To ensure a quick and adequate solution and to ensure your users get the most out of their Microsoft 365. But did you know that ‘bad calls’ are also an excellent indicator of other problems arising? Precisely because it is so noticeable!? Like the canary in the mine, bad calls can be early indicators of problems. Problems that might otherwise not have been noticed for a while but can have a big impact on productivity and satisfaction. Join this session by Christoph Adler to learn how true Microsoft Teams call quality analytics helped other organizations troubleshoot bad calls and identify and fix problems that impacted Teams calls or the use of Microsoft365 in general. See what it can do to keep your users happy and productive! In this session we will cover - Why CQD data alone is not enough to troubleshoot call problems - The importance of attributing call problems to the right call participant - What call quality analytics can do to help you quickly find, fix-, and prevent problems - Why having retrospective detailed insights matters - Real life examples of how others have used Microsoft Teams call quality monitoring to problem shoot problems with their ISP, network, device health and more.

Why Teams call analytics are critical to your entire business

panagenda

Tata AIG General Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

Partners Life - Insurer Innovation Award 2024

The Digital Insurer

A Domino Admins Adventures (Engage 2024)

Gabriella Davis

The Good, the Bad and the Governed - Why is governance a dirty word? David O'Neill, Chief Operating Officer - APIContext Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

apidays

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

Join our latest Connector Corner webinar to discover how UiPath Integration Service revolutionizes API-centric automation in a 'Quote to Cash' process—and how that automation empowers businesses to accelerate revenue generation. A comprehensive demo will explore connecting systems, GenAI, and people, through powerful pre-built connectors designed to speed process cycle times. Speakers: James Dickson, Senior Software Engineer Charlie Greenberg, Host, Product Marketing Manager

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

DianaGray10

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc

presentation ICT roal in 21st century education

jfdjdjcjdnsjd

Strategies for Landing an Oracle DBA Job as a Fresher

Remote DBA Services

Artificial Intelligence Chap.5 : Uncertainty

Khushali Kathiriya

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Edi Saputra

Axa Assurance Maroc - Insurer Innovation Award 2024

The Digital Insurer

Increase engagement and revenue with Muvi Live Paywall! In this presentation, we will explore the five key benefits of using Muvi Live Paywall to monetize your live streams. You'll learn how Muvi Live Paywall can help you: Monetize your live content easily: Set up pay-per-view access to your live streams and start generating revenue from your content. Increase audience engagement: Provide exclusive, premium content behind the paywall to keep your viewers engaged. Gain valuable viewer insights: Track viewer data and analytics to better understand your audience and tailor your content accordingly. Reduce content piracy: Muvi Live Paywall's security features help protect your content from unauthorized distribution. Streamline your workflow: The all-in-one platform simplifies the process of managing and monetizing your live streams. With Muvi Live Paywall, you can take control of your live stream monetization and create a sustainable business model for your content. Learn more about Muvi Live Paywall and start generating revenue from your live streams today!

Top 5 Benefits OF Using Muvi Live Paywall For Live Streams

Roshan Dwivedi

MySQL Webinar, presented on the 25th of April, 2024. Summary: MySQL solutions enable the deployment of diverse Database Architectures tailored to specific needs, including High Availability, Disaster Recovery, and Read Scale-Out. With MySQL Shell's AdminAPI, administrators can seamlessly set up, manage, and monitor these solutions, ensuring efficiency and ease of use in their administration. MySQL Router, on the other hand, provides transparent routing from the application traffic to the backend servers in the architectures, requiring minimal configuration. Completely built in-house and supported by Oracle, these solutions have been adopted by enterprises of all sizes for their business-critical applications. In this presentation, we'll delve into various database architecture solutions to help you choose the right one based on your business requirements. Focusing on technical details and the latest features to maximize the potential of these solutions.

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Miguel Araújo

Wiki[mp]edia data sources & the MediaWiki API

1. Wiki[mp]edia data sources & the MediaWiki API Brianna Laugher for #melhack November 2009

2. ...

3. Wikipedia 13M articles total 3M+ articles in English 240+ languages Simple English!

4. {{coord|37|48|49|S|144|57| 47|E|type:city_region:AU-VIC| display=inline,title}} stable.toolserver.org/geohack/ wiki.toolserver.org/view/GeoHack

5. {{Infobox Company |name = Lonely Planet |logo = |type = [[United Kingdom|British]] [[Government-owned company|government-owned]] (subsidiary of [[BBC Worldwide]]) |genre = [[Guide book|Travel guides]] |foundation = 1972 |founder = Tony Wheeler Maureen Wheeler |location_city = [[Footscray, Victoria]] |location_country = [[Australia]] |location = |origins = |key_people = Matt Goldberg (Global [[CEO]]) |area_served = Worldwide |industry = [[Multi media]] |products = Travel [[guidebook, digital applications, online travel community]] |services =

6. Wikimedia Commons commons.wikimedia.org Multilingual 5M+ files “Self-created”, PD, Flickr Predominantly photographs, but also diagrams, maps, flags

8. Wiktionary 5M+ entries 170+ languages 13 languages > 100K entries French biggest at 1.5M (English second at 1.4M)

9. JavaScript Wiktionary lookup plugin for third parties: http://bawolff.blogspot.com/2009/10/introducing- wiktionary-lookup-now-for.html http://en.wiktionary.org/wiki/Wiktionary:Parsing

10. MediaWiki structure  Users  Logs  Pages, subpages, talk pages  Links, backlinks  Templates  Categories

11. MediaWiki markup The only thing that completely understands it is MediaWiki :(

12. Database dumps XML download.wikimedia.org OR Amazon Public Data Sets meta.wikimedia.org/wiki/ Data_dumps

13. DBpedia Community project extracting structured data from Wikipedia and making it available Can download data sets or query them online Ontology++ e.g. dbpedia.org/page/Lonely_Planet

14. MediaWiki API mediawiki.org/wiki/API en.wikipedia.org/w/api.php Client libraries!

15. mwclient Python library for accessing MediaWiki APIs

16.

17. Toolserver toolserver.org Server for community-developed plugins, addons, extensions, stats and hacks – tools Tools often explicitly implements implicit editing community standards (“community API”)

18. TemplateTiger toolserver.org/~kolossos/templatetiger/ For a few dozen Wikipedia languages, & Wikimedia Commons Lets you query templates very much like SQL

19. Thanks! identi.ca/pfctdayelise blaugher@wikimedia.org.au Logos and screenshots may be copyright their respective owners Slides are otherwise © Brianna Laugher

Wiki[mp]edia data sources & the MediaWiki API

Recomendados

Recomendados

Más contenido relacionado

Similar a Wiki[mp]edia data sources & the MediaWiki API

Similar a Wiki[mp]edia data sources & the MediaWiki API (20)

Más de Brianna Laugher

Más de Brianna Laugher (20)

Último

Último (20)

Wiki[mp]edia data sources & the MediaWiki API