Greg Kempe

Announcing Slaw, a lightweight library for Akoma Ntoso Acts

20 Nov 2014

Today I'm announcing Slaw, a lightweight Ruby gem library for producing and rendering legislative acts using the Akoma Ntoso XML format. It simplifies the process of parsing PDF and plain-text documents into Akoma Ntoso XML and rendering that XML into well-formatted HTML.

Slaw is used by openbylaws.org.za to render the XML content of the by-laws into HTML. It's also used by steno.openbylaws.org.za to take PDF and plain-text versions of the by-laws and generate the corresponding XML which powers the website.

Where Slaw Comes From

When a started Open By-laws South Africa I needed a tool to take the PDF versions of the by-laws available on the websites of South African municipalities -- such as Cape Town -- and turn them into decent looking webpages that are easy to read, navigate and share.

The Akoma Ntoso XML format is designed to capture the structure and metadata of an piece of legislation and met my needs. But I needed a way of generating the XML from the PDF without having to write XML by hand.

So I created Slaw. It does the following:

  1. extracts plain text from PDF documents and cleans it up
  2. parses that text into an XML format which captures the content and structure of the legislation
  3. transforms the XML into HTML which, when paired with a CSS stylesheet, produces legislation that is well formatted and easy to read.

Show Me the Code

Enough chit chat, here's how to use Slaw. We're going to convert this PDF of Joburg's Metered Taxi by-law into HTML.

First, install the gem:

$ gem install slaw

You will also need xpdf to do the PDF extraction. On a Mac you can use Homebrew.

$ brew install xpdf

Now fire up irb and lets grab the plain text from the PDF (which you should download):

require 'slaw'
# extract text from a PDF file and clean it up
extractor = Slaw::Extract::Extractor.new
text = extractor.extract_from_pdf('dogs-and-cats-source-enacted.pdf')

If you puts text, you'll get something like this:

Slaw has managed to find and remove the table of contents and has done a reasonable job of fixing line wraps, etc. Now let's parse that text into a by-law object, backed by XML.

generator = Slaw::ZA::BylawGenerator.new
# tell the parser that section numbers come after the section title
generator.parser.options = {section_number_after_title: true}
bylaw = generator.generate_from_text(text)
# set the title
bylaw.title = 'By-laws relating to Dogs and Cats, 2005'

We can see the XML:

puts bylaw.to_xml(indent: 2)

Now let's transform that into HTML:

# render the by-law as HTML, using / as the root for relative URLs
renderer = Slaw::Render::HTMLRenderer.new
html = renderer.render(bylaw.doc, '/')

and if we link it to a stylesheet from openbylaws.org.za, we get this:

CITY OF JOHANNESBURG METROPOLITAN MUNICIPALITYMETERED TAXI, MINIBUS, MIDIBUS AND BUS BY-LAWSThe Municipal Manager of the City of Johannesburg Metropolitan Municipality hereby, in terms of Section 13(a) of the Local Government: Municipal Systems Act, 2000 (Act No 32 of 2000), publishes the Metered Taxi, Minibus, Midibus and Bus By-laws for the City of Johannesburg Metropolitan Municipality, as approved by its Council and as concurred with by the Premier: Gauteng Province in terms of section 80A of the National Road Traffic Act, 1996 (Act 89 of 1996), as set out hereunder.CITY OF JOHANNESBURG METROPOLITAN MUNICIPALITYMETERED TAXI, MINIBUS, MIDIBUS AND BUS BY-LAWS

Chapter 1
Interpretation

1. Definitions

(1)In these By-laws, any word or expression that has been defined in the Gauteng Public Passenger Road Transport Act, 2001 (Act No. 7 of 2001) has that meaning and, unless the context otherwise indicates –"authorised official" means any official of the Council who has been authorised by it to administer, implement and enforce the provisions of these By-laws;"Council" means – (a)the Metropolitan Municipality of the City of Johannesburg established byProvincialNotice No. 6766 of 2000 dated 1 October 2000, as amended,exercising its legislative and executive authority through its municipal Council; or (b)its successor in title; or(c)a structure or person exercising a delegated power or carrying out an instruction, where any power in these by-laws has been delegated or sub-delegated, or an instruction given,as contemplated in section 59 of the Local Government: Municipal Systems Act, 2000 (Act No. 32 of 2000); or (d)a service provider fulfilling a responsibility under these by-laws, assigned to it in terms of section 81(2) of the Local Government: Municipal Systems Act, or any other law, as the case may be."Johannesburg Metropolitan Police Department" means the municipal police department of the Council established in terms of section 64A of the South African Police Service Act, 1995 (Act No. 68 of 1995);"lift club" means any club of which every member shall, for no direct or indirect reward, have a turn to convey or cause to be conveyed by means of a motor car, the members of such a club or other persons designated by such members, to or from specified places for a specific purpose;"Medical Officer of Health" means a person appointed as such under section 22 or 25 of the Health Act, 1977 (Act No. 63 of 1977);"prescribed fee" means a fee determined by the Council by resolution in terms of section 10G(7)(a)(ii) of the Local Government Transition Act, 1993 (Act No. 209 of 1993), or any other applicable legislation."public road" means a public road as defined in the National Road Traffic Act, 1996;"queue marshal" means a person designated by a taxi association to regulate minibus taxi-type services; and"rank" means a facility set aside by the Council for use by public passenger road transport;"cab licence" means a payment that may be charged annually as a once-off charge, and which gives road-based public transport operators the “right” or “permission” to operate within the Council’s municipal boundary."road-based public transport operators" means operators of metered taxis, minibuses, midibuses and buses.(2)If any provision in these by-laws vests or imposes any power, function or duty of the Council in or on an employee of the Council and such power, function or duty has in terms of section 81(2) of the Local Government: Municipal Systems Act, 2000 or any other law been assigned to a service provider, the reference in such provision to such employee must be read as a reference to the service provider or, where applicable, an employee of the service provider authorised by it.CHAPTER 2

2. Driver to take shortest route

(1)A driver of any metered taxi must, while the metered taxi is hired, drive to the passenger’s destination along the shortest route, unless another route is agreed on or directed by the passenger.(2)A metered taxi driver must have a current map of the municipal area in his or her possession, which must be made available by the driver to a passenger on request.

3. Driver to keep engagement

(1)A driver of any metered taxi must convey a passenger and his or her personal effects to the destination agreed upon between the passenger and the driver.(2)Should the driver of a metered taxi for any reason whatsoever, be unable to convey the passenger and the passenger’s personal effects to the agreed destination, such driver must take all reasonable steps to arrange another metered taxi for the passenger, or let the passenger arrange for transport to get to his or her destination.

4. Operation of taximeter

(1)The driver of a metered taxi fitted with a taximeter must, as soon as the driver arrives at the point where his or her hiring commences and not sooner, set the taximeter in motion, and must upon the termination of hiring immediately stop the taximeter from recording.(2)Upon the occurrence of any stoppage not caused by traffic congestion or by the action or request of any passenger, the said driver must for the duration of such stoppage stop the taximeter from recording.(3)The owner of a metered taxi must ensure that the provisions of subsections (1) and (2) and the minimum or maximum fare as determined in terms of section 8(2) of the Gauteng Public Passenger Road Transport Act, 2001, if any, are affixed to the interior of his or her taxi, in such a position that they can be easily read by a passenger in the taxi.

Chapter 3
Buses

5. Stopping places

No driver of a bus, as defined in the National Land Transport Transition Act, 2000 (No. 22 of 2000), may stop it for the purpose of picking up or setting down any passenger, except at a stopping place designated by the Council.

6. Entering and alighting from a bus

A prospective passenger of a bus, as defined in the National Land Transport Transition Act, 2000 (No. 22 of 2000), may only enter or alight from a bus at a stopping place designated by the Council.

7. Driver to stop at stopping places

The driver of a vehicle engaged in a public passenger road transport service, which at the time is not carrying the maximum number of passengers the vehicle is lawfully entitled to carry, must stop at any designated stopping place if a prospective passenger is waiting at such stopping place.

Chapter 4
General

8. Parking of metered taxi, minibus, midibus or bus

No person may park a metered taxi, minibus, midibus or bus on any public road for the purpose of providing a public passenger road transport service, except in an exclusive parking bay, marked by a road traffic sign as prescribed in terms of the National Road Traffic Act, 1996 (Act No. 93 of 1996) for that vehicle.

9. Parking at places of entertainment or funeral

Notwithstanding the provision of section 8 of these By-laws, a metered taxi may park on a public road for the purpose of providing a metered taxi service, where a party or private entertainment is in progress or from which any funeral or wedding procession is about to start.

10. Engagement of passengers

(1)No driver of a metered taxi, minibus, midibus or bus, or any other person, may by using force or a threat, or in a clandestine manner or by any other means, prevent or seek to prevent any person from hiring any other metered taxi, minibus, midibus or bus or prevent or seek to prevent the driver of such other metered taxi, minibus, midibus or bus from obtaining or conveying a passenger or a load.(2)No person may use force, a threat, or any clandestine or other method, to prevent or attempt to prevent any person from participating in a lift club.(3)The driver or conductor of a metered taxi, minibus, midibus or bus may not use a hooter or sounding device to attract the custom of potential passengers or customers.

11. Failing or refusing to pay or attempting to evade payment of the fare due

No hirer of, or passenger in or on a vehicle engaged in a public passenger road transport service, may fail or refuse to pay any fare due by such hirer or passenger.

12. Furnishing of name and address by person conveyed in or on metered taxi, minibus, midibus or bus

Any person hiring, or conveyed in or on, a vehicle engaged in a public passenger road transport service, who has failed or refused to pay any fare due by him or her, must when requested to do so by the driver, state his or her correct name and address.

13. Conveyance of filthy or diseased persons

(1) A driver of a vehicle engaged in a public passenger road transport service may refuse to convey or carry - (a)any person who is obviously in a state of filth or obviously suffering from any contagious disease; or(b)any dead animal except animals or poultry intended for human consumption if the animal or poultry is properly wrapped.(2)No person who has another person in his or her care who to his or her knowledge has been exposed to, or contaminated with, any contagious disease, may place such person in any metered taxi, minibus, midibus or bus.(3)No person who is obviously in a state of filth or obviously suffering from any contagious disease may enter any metered taxi, minibus, midibus or bus or, having entered, remain upon such vehicle after being requested by the driver or conductor thereof to leave the vehicle.

14. Disinfection of metered taxi, minibus, midibus or bus

(1) The owner, driver, conductor or any other person in charge of a vehicle engaged in a public passenger road transport service must take immediate steps as soon as it comes to his or her knowledge that - (a)any person suffering from a contagious disease; or(b)the body of any person who has died of such disease; or(c)anything which has been exposed to or contaminated with such disease,has been conveyed in or upon such vehicle engaged in a public passenger road transport service to report the matter to the Medical Officer of Health.(2)Any owner, driver, conductor or other person referred to in subsection (1) must carry out every instruction issued by the Medical Officer of Health with regard to the disinfection of such vehicle engaged in a public passenger road transport service.

15. Driver’s right to refuse to convey passengers

(1)The driver of a vehicle engaged in a public passenger road transport service may, if he or she so decides, or at the request of any passenger, refuse to convey any person who is obviously in a state of intoxication or who is noisy or rowdy or otherwise misbehaving himself or herself.(2)No person referred to in subsection (1), may remain in or upon such vehicle engaged in a public passenger road transport service, after having been requested by the driver or conductor thereof to leave the vehicle engaged in a public passenger road transport service.

16. Property left in metered taxi, minibus, midibus or bus

(1) If any property left in a vehicle engaged in a public passenger road transport service is not claimed within 24 hours after it has been discovered in such vehicle engaged in a public passenger road transport service, the driver or conductor of the vehicle must – (a)if he or she belongs to a taxi association, take such property to the nearest office of such association;(b)if he or she uses a bus depot for the purposes of the business in which he or she is engaged, take such property to such depot; or(c)if he or she does not belong to a taxi association or use a bus depot for the purposes of the business concerned, take such property to the Johannesburg Metropolitan Police Department,and obtain a receipt from the person with whom the property is deposited, or the officer on duty at the Johannesburg Metropolitan Police Department, as the case may be.(2)If the property referred to in paragraphs (a) and (b) of subsection (1) is not claimed within seven days of its receipt in the office of the relevant taxi association or bus depot, the person with whom it was deposited must take it to the Metropolitan Police department, and there deposit it with the officer on duty, who must issue a receipt for such property to the person depositing it.

17. Possession of dangerous or offensive articles

(1)If the driver or conductor of a vehicle engaged in a public passenger road transport service reasonably suspects that any passenger is in possession of any dangerous or offensive article, except a fire-arm as described in subsection (2), the driver or conductor or any other passenger may request the first mentioned passenger to hand such article to the driver or conductor.(2)If a passenger is in possession of a fire-arm, the driver or conductor may request the passenger to display a valid licence for such fire-arm or, if the passenger is required to carry the fire-arm as a member of the national or a municipal police service established in terms of the South African Police Service Act, 1995, or as a member of the National Defence Force established under the Defence Act, 1957 (Act No. 44 of 1957), the driver or conductor may request the passenger to display the current identity document which was issued to the passenger by such service or force.(3)If the passenger refuses to hand the article referred to in subsection (1), or fails to display the licence or identity document referred to in subsection (2), to the driver or conductor, the driver may refuse to convey the passenger.(4)The article referred to in subsection (1) must be returned to its owner at the conclusion of his or her journey.

18. Cleanliness

The driver or conductor of any vehicle engaged in a public passenger road transport service must be clean and neatly dressed at all times while conveying a passenger, and must treat every passenger politely and with respect.

19. Queue marshal

(1)A queue marshal at any rank must be clearly identifiable as to his or her employer and must display his or her name in a conspicuous manner on his or her clothing below the left shoulder.(2)A queue marshal must discharge his or her duties in a courteous and polite manner and show respect to every passenger.(3)Where a queue marshal is controlling the entry of passengers onto a metered taxi, minibus, midibus or bus, he or she must not allow more than the number of passengers permitted by law, to enter such metered taxi, minibus, midibus or bus.

20. Clean vehicle

The owner and the driver of any metered taxi, minibus, midibus or bus must keep the vehicle clean and in good condition at all times while engaged in public passenger road transport services.

21. Offences and penalties

Any person who – (a)contravenes or fails to comply with any provisions of these By-laws;(b)fails to comply with any notice issued in terms of these By-laws;(c)fails to comply with any lawful instruction given in terms of these By-laws; or(d)who obstructs or hinders any authorised official of the Council in the execution of his or her duties under these By-laws,is guilty of an offence and liable on conviction to a fine or in default of payment to imprisonment for a period not exceeding six months and in the case of a continuing offence, to a further fine not exceeding R50, or in default of payment to imprisonment not exceeding one day, for every day during the continuance of such offence, after a written notice has been issued by the Council, and served on the person concerned, requiring the discontinuance of such offence.

22. Repeal of by-laws

The by-laws listed in Schedule 1 are hereby repealed.

23. Short title

These By-laws are called the Metered Taxi, Minibus, Midibus and Bus By-laws, 2003.

24. Charging of the cab licence

The City may charge a cab licence to operators of all modes of road-based public transport on an annual basis as a once-off payment. The cab licence may also be reviewed annually.

Not Quite Perfect, but Good Enough

As you can see above, the process isn't perfect. For example, it's struggled with the definition of council and doesn't like the fact that Chapter 2 doesn't have a title. However, it's got 90% of the way there with no input from us. It has correctly nested lists and discovered section and chapter headings, even definitions of terms.

We built steno.openbylaws.org.za to capture the remaining 10%. It gives the user the opportunity to straighten up a few things before converting it to XML. It also lets you set the publication date and some other meta-data. Using Steno, we can capture a new, multiple-page by-law in 10-20 minutes, which is far less than retyping it from scratch.

Steno is also available on GitHub.

How Slaw Works

Slaw uses Treetop to compile a grammar into a backtracking parser. The parser builds a parse tree, each node of which knows how to serialize itself in XML format. There's more information in the README.md and I'll save the details for another blog post.

Use, Fork and Contribute

Slaw is licensed under an MIT license and we welcome pull requests and discussion. I'm excited to see how others can make use of this and slowly push forward the boundaries of the Free Access to Law movement around the world.