Open Knowledge Foundation Blog

The last six months have seen significant developments on our Open Shakespeare project, many of which have are reflected on the website: http://www.openshakespeare.org/

The most major advance is the availability of new HTML and PDF editions of the texts, see, for example, these versions of Twelfth Night:

• http://www.openshakespeare.org/resource/view/92/twelfth-night-moby/
• http://www.openshakespeare.org/pdf/twelfth_night_moby.pdf

We’ve also made improvements to multiview, cleaned up the web interface, revamped the domain model (proper Work/Edition/Resource distinction), and much more!

Going forward our main efforts are, on the “tech” side, to integrate a new (javascript) annotation system, and on the content side it’s developing our open “critical edition” (an effort now being led by some students at Oxford and Cambridge).

We’re also holding a regular Open Shakespeare (virtual) meetup every other Saturday @ 4pm (London time) with the next one this coming Saturday (the 24th). All are welcome, so if you’re interested in Shakespeare why not drop in — details for how to participate are on the project wiki page.

Share This

We’ve been thinking for a while that it would be a nice addition to the Open Shakespeare project to produce an “Open Shakespeare Edition” of the Bard’s works.

By an ‘Edition’ we meant something designed as a book and suitable for printing: so an elegant title page, relevant front-matter, properly typeset text etc. This could then be downloaded by users and printed or even offered in dead-tree version directly using print-on-demand.

Recently, we’ve made a start on this endeavour using the moby XML sources, xsl and latex. An example of the results can be seen at:

http://www.openshakespeare.org/images/twelfth_night-v0.2.pdf

As a cursory look at that will show, while the body of the play doesn’t look too bad, the front-page could do with improvement (and the front-matter generally needs some planning). So, questions for readers:

1. Anyone out there with design skills or suggestions who could help us out?

   • Would it make sense to run a design competition?

2. What kind of general look should we go for? For example, should we go for:

   • Ultra traditional (but perhaps with some mods e.g. replacing the standard ‘copyright’ section with something about open knowledge)
   • Something irreverent, for example along the lines of the sketch on http://okfn.org/wiki/ShakespeareBookDesign

Any ideas or suggestions post a comment or drop us a line we’d love to know what you think.

Share This

See http://pypi.python.org/pypi/shakespeare/0.6 which includes full installation instructions. We’ve also reorganized the sites so that the news/blog is here at http://blog.openshakespeare.org/ and the Shakespeare package web interface is at http://www.openshakespeare.org.

Main changes include:

• Major refactoring of internal code to be cleaner and simpler
• A new cleaner and reorganized web interface
• Search support via Xapian: http://www.openshakespeare.org/search/
• Statistical analysis and graphing
  • By word: http://www.openshakespeare.org/stats/word/love/
  • By text: http://www.openshakespeare.org/stats/text/hamlet_gut
• Start on Open Milton

Share This

Another 3 pages (4600 words) are up from the EB 11 Entry on Shakespeare covering most of Shakespeare’s plays in chronological order. Current material can be found on:

Encyclopaedia Britannica 11th Edition page

Source version (plain text in subversion) can be found at:

http://knowledgeforge.net/shakespeare/svn/trunk/shksprdata/ancillary/britannica-11th.txt

Share This

After a fairly quiet period over the last 6 months development will be hotting up again thanks to discussion at Open Knowledge 2008 and the involvement of Iain Emsley (who will be focusing especially on a sister Milton project). To kick this off we’re planning a mini-hackathon:

• Wiki page: (sign up here) http://www.okfn.org/wiki/MiniEvents
• When: Saturday 26th of April. Start at 1400 and run until ~ 1900
• How long: Whatever time you can spare. Be it an hour or the whole afternoon.
• How to join in: log in to the irc channel, announce yourself, and then just crack on with one of the work items (see below)
  • irc channel: #okfn on irc.oftc.net
• What: plan and work on Open Shakespeare / Milton
  • trac: http://knowledgeforge.net/shakespeare/trac/roadmap
  • Current tickets: http://knowledgeforge.net/shakespeare/trac/report/1
  • For those not inclined to coding there’s plenty else to do. In particular we need to finish off proof editing Britannica entry, see http://wiki.okfn.org/p/Open_Shakespeare/Britannica

Share This

We’ve completed the proofing and correcting of the first 5 pages of Shakespeare’s Entry from the 1911 Encyclopaedia Britannica. This is quite a bit of material (those EB pages are big) and includes full biography and a listing of plays. We’re posting this material on this site on Encyclopaedia Britannica 11th Edition page and will add to it as more material gets processed.

Share This

Since the previous post we’ve succeeded in using tesseract and we now have a nice plain text version of the EB entry on shakespeare:

http://knowledgeforge.net/shakespeare/svn/trunk/shksprdata/ancillary/britannica-11th.txt

What we now need to do is ‘proof’ this to correct the OCR errors. This kind of think is perfect for distributed volunteers so if you’d like to help out just step up and starting correcting with one of the sections. To make it especially easy for people to make edits the text has in a temporary location on the Open Knowledge Foundation wiki (only the first five pages for the time being):

http://wiki.okfn.org/p/Open_Shakespeare/Britannica

Share This

One of next things we want to do for open shakespeare is provide an open introduction for to his works. The obvious idea for this was to use the Shakespeare entry in the 11th ed of the Encyclopaedia Britannica as detailed in this ticket:

http://p.knowledgeforge.net/shakespeare/trac/ticket/24

We’ve now written code to grab the relevant tiffs off wikimedia:

http://p.knowledgeforge.net/shakespeare/svn/trunk/src/shakespeare/src/eb.py

You can also find them online (28 pages) starting at:

http://upload.wikimedia.org/wikipedia/commons/scans/EB1911_tiff/VOL24%20SAINTE-CLAIRE%20DEVILLE-SHUTTLE/ED4A800.TIF

Next step is to then OCR this stuff (after that we can move on to proofing whether by ourselves or via http://pgdp.net). When we first had a stab at this back in April we tried using gocr. Unfortunately the results were so bad that they were unusable. Recently an old ocr engine of HP’s has been released as open source under the name of tesseract:

http://code.google.com/p/tesseract-ocr/

We’re going to have a go using this — though if there is anyone out there with access to an alternative system we’d love to hear about it.

Share This

A new version of open shakespeare is out. Get it via the code page:

http://www.openshakespeare.org/code/

Changelog

• Annotation of texts (js-based in browser) (ticket:20, ticket:21) (http://www.openshakespeare.org/2007/04/10/annotation-is-working/)
• Switch to unicode for internal string handling (resolves ticket:23: some texts breaking the viewer)
• Add functional tests for the web interface (ticket:11)
• Substantial improvements to speed of concordance (ticket:22) (http://www.openshakespeare.org/2007/01/03/improvements-to-the-concordance/)
• Switch to genshi templates from kid
• Switch to plain WSGI from cherrypy

Outstanding Issues

• Annotation cannot handle long texts because of javascript performance issues

About Open Shakespeare

A full open set of Shakespeare’s works along with anciallary material, a variety of tools and a python API.

For more information see the about page:

http://www.openshakespeare.org/about/

Get involved: http://www.openshakespeare.org/participate/

Mailing list: http://lists.okfn.org/mailman/listinfo/okfn-discuss/

Share This

After another push over the last few days I’ve got the web annotation system for Open Shakespeare operational (we’ve been hacking on this on and off since back in December).

To see the system in action visit:

http://demo.openshakespeare.org/view?name=phoenix_and_the_turtle_gut&format=annotate

Quite a bit of effort has been made to decouple the annotation system from Open Shakespeare so that it can be easily reused elsewhere. You can find the code for the annotation system (nicknamed annotater) here:

http://p.knowledgeforge.net/shakespeare/svn/annotater/trunk/

There are still some substantial issues with the Open Shakespeare implementation the most obvious of which are:

a) large texts bring the javascript to its knees ((The Phoenix and the Turtle is the shortest of Shakespeare’s works which is why I’m using it).

b) security/user authentication for annotation adding/editing/deleting

But the basic system is working.

Share This