Tag: OIA

Official information kept secret too long

Official information kept secret too long

Official information is being kept secret for longer than it should be. For the past few months, I have been gathering and analysing data from 12 government agencies, looking at how they handle requests made under the Official Information Act.

My findings reflect what many have observed more anecdotally: responses to requests for information are often sent at the very last minute, and seem to often be delayed unnecessarily. Though it can be difficult to demonstrate that a response was not sent “as soon as reasonably practicable” — a requirement under the law — in any particular case, looking at a larger data set reveals some of the strategies used to delay the release of official information.

You can read my article here: Official information kept secret too long

OIA accessibility follow-up

OIA accessibility follow-up

What will be done about the Official Information Act’s accessibility problem?

Late last year, I published an open letter on the accessibility of the Official Information Act. I gave examples of inaccessible responses to requests under the OIA, and suggested a solution. I sent this letter to the Ombudsman, and later also forwarded it to Associate Minister of State Services (Open Government) Clare Curran and to Minister for Disability Issues Carmel Sepuloni.

I’ve now heard back from the latter two, and want to share their responses so others interested in this topic can read what’s being done to address it.

I’d also lodged an OIA request with the Ministry of Defence last year, as their Briefing to Incoming Minister document was released as a text-based PDF with watermarks and redaction. I’d asked them what software they used to produce this, and the associated costs, so I’d be able to give a specific recommendation to agencies who send me inaccessible responses in the future.

I’ve received a response to this request as well. As with the responses from Clare Curran and Carmel Sepuloni, you can find the text and PDF link to this response below.

Unfortunately, and somewhat ironically, all three responses were received in the form of image-based PDFs of documents that had been printed, signed, and then scanned. This is the primary accessibility issue I’d raised in my letter regarding OIA responses, as this format cannot be easily converted to text and is inaccessible to people who rely on screen readers. A couple of the responses also contained links that I’ve had to type out by hand instead, due to this inaccessible format. I’ve made the original PDFs available, and also converted them to text (by hand, so please assume any mistakes are mine), which you can find below:

Response from Clare Curran, Associate Minister of State Services (Open Government). Original PDF

Dear Mark Hanna

OIA Accessibility

Thank you for your email on 30 November 2017, regarding the accessibility of information released to requesters under the Official Information Act 1982 (OIA). I note your concerns about the format some agencies choose to provide information in, which can render that information unsearchable by the recipient.

As you are aware, section 16(2) of the Official Information Act provides that information can be made available in a way preferred by the requester, unless there is good reason identified by the agency not to do this. In the first instance it is for the requester to ask the agency to make information, and in particular data sets available in a searchable format. If the agency refuses then this can be the subject of a complaint to the Ombudsman.

The State Services Commissioner has been delegated the functions under section 46 of the OIA by the Ministry of Justice. This function includes providing advice and guidance to agencies to act in accordance with the OIA. SSC has established an official information work programme which includes developing guidance for agencies. I think that the issues you have raised are worthy of further consideration.

I have asked the State Services Commission to consider whether accessibility of information should be part of the official information work programme that the Commission is undertaking.

Yours sincerely

Hon Clare Curran
Associate Minister of State Services (Open Government)

Response from Carmel Sepuloni, Minister for Disability Issues. Original PDF

Dear Mr Hanna

Thank you for your email dated 30 November 2017, regarding accessibility issues with Official Information Act (OIA) practices and across government more generally. The Government is committed to being open and transparent and we expect the same of the public service.

I have been advised that the Official and Parliamentary Information team within the Ministry of Social Development will soon utilise more sophisticated redaction software. This software will enable the Ministry of Social Development to send and publish OIA responses and documents that are text searchable and text recognisable for those requesters who use reading software.

Providing OIA responses in accessible formats should be standard practice as part of an open and transparent government. To this end, the Ministry of Social Development is leading a work programme to increase the accessibility of information and communications across government agencies. This will involve providing clear and practical advice to support government agencies to meet accessibility requirements for all their communications with the public. Participating Chief Executives have signed up to an Accessibility Statement to commit to this. You can find more information about this here: www.odi.govt.nz/nz-disability-strategy/outcome-5-accessibility/

Thank you for writing. I hope this information is helpful.

Ngā mihi

Hon Carmel Sepuloni
Minister for Disability Issues

Response from Ministry of Defence. Original PDF

Dear Mr Hanna

RESPONSE TO YOUR OFFICIAL INFORMATION REQUEST

Thank you for your email of 7 December 2017 regarding the software used to create the redacted and watermarked 2017 Briefing to the Incoming Minister of Defence, published on the Beehive website.

The Ministry of Defence recognises that publishing a document online as a text-searchable PDF file makes is easier to access. We prepare material for release in this form wherever possible, and endeavour to provide alternative formats on request for non-searchable documents, such as older files.

In order to produce searchable, redacted and watermarked files, the source documents are generally prepared using Microsoft Office software and then exported as searchable PDF files. Adobe Acrobat Professional software is then used to make redactions and add watermarks.

The Ministry has an agreement with the New Zealand Defence Force on software licensing. The Defence Force pays for an all-inclusive software bundle and is charged a flat rate. They distribute the licenses for Adobe Acrobat Professional as needed by staff. Please refer to the Adobe Acrobat website for costs: https://acrobat.adobe.com/nz/en/acrobat/pricing.html. The cost to agencies will be affected by volume. Assuming multiple licenses are purchased, the Adobe website indicated pricing of A$22.99 per seat per month (about NZ$25 at present exchange rates) or A$275.86 per year (about NZ$300).

Under section 28(3) of the Official Information Act 1982 you have the right to request the Ombudsman to investigate and review this response.

Yours sincerely

Helene Quilter
Secretary of Defence

It’s not clear yet whether or not accessibility of information will be part of the SSC’s official information work programme that Clare Curran mentioned. Though I’m hopeful that it will be, I’m not sure how likely this is to effect change.

I strongly agree with Carmel Sepuloni’s statement that “Providing OIA responses in accessible format should be standard practice as part of an open and transparent government”, and hope that she will do her best to ensure our government meets that standard. I’m cautiously optimistic, though the fact that this response itself was delivered in an inaccessible format isn’t encouraging.

For the foreseeable future, it sounds like the best course of action will be to ensure that all OIA requests specify that the response should be delivered in an accessible, searchable format. I have included this recommendation in the OIA Guide that I published last year.

If I receive inaccessible PDF responses in the future, I now intend to direct the agency to the Ministry of Defence’s response regarding the software they use.

OIA Accessibility

OIA Accessibility

The Official Information Act has an accessibility problem.

I wrote recently about asking the government for information, having just published a guide to using the OIA. The OIA is a powerful tool, but it can be limited by how government agencies choose to follow it. One particular limitation that comes up again and again is accessibility.

There is a hashtag on Twitter that’s often used by journalists and activists, myself included, to talk about issues with the Official Information Act: #fixtheOIA

Today Nikki Macdonald, senior feature writer at the Dominion Post, started a discussion about a widespread and rather infuriating practice for responding to requests for official information:

In response to a recent request she made under the OIA, she received a PDF of a scanned document that included a large amount of tabular data.

The issue she mentions is one I have seen many times. It is common practice to take a document that will be released under the OIA, print it, then scan it back into a PDF, then send that PDF instead of the original document.

Printing and scanning documents allows for them to be signed, which is typically done with OIA response letters (though not with documents released alongside them). PDFs cannot be altered by the recipients, so the information is kept intact as it was released.

Also, perhaps most importantly, because these PDFs come from scanned documents they don’t contain any text. Instead, they just contain images of text. This means there is absolutely no way the recipient will be able to view information that has been withheld from released documents.

But releasing documents as images of text has a huge drawback too: accessibility

Most obviously, this will affect requestors who are blind or have impaired vision. If you rely on a screen reader to read these documents, it won’t be able to read a PDF that’s full of images. It’s a pain in the ass for those of us who are fully sighted as well.

There is software that can be used to convert images of text back into text, but it doesn’t always work perfectly, and often these image-based PDFs will include tables of data that are only really useful once converted into a spreadsheet. And, frankly, knowledge of how to use specialised software shouldn’t be a requirement for using the OIA to get usable information.

This isn’t the only accessibility issue with the OIA. Government agencies have also been known to do things like refuse to release documents online for unclear “security reasons“, instead telling the requestor that they must pay to have it posted to them.

Recently, Sam Warburton has been documenting a particularly trying OIA response on Twitter. First, he was sent documents that were locked by a password, which prevented him from copying text out of them:

He then made a second OIA request asking for all correspondence regarding the first request, so he could get to the bottom of how this bizarre twist had come about. The response was, well, not very accessible. He was sent 249 pages of printed documents, with the excuse being that it was “too big” to send digitally:

I think we should be able to expect better. Far better, in fact. And I’ve written the following open letter to the Ombudsman in the hope that this aspect of using the OIA can be improved:

Tēnā koe,

There has been a discussion on the “#fixtheOIA” hashtag on Twitter started by Nikki Macdonald today about the common practice of agencies sending responses to requests for official information in PDF format, where the contents of the PDF must be either transcribed by hand or converted using specialised software before they are useful.

I’ve observed this practice across several agencies, including those I have most commonly made requests to: New Zealand Police, the Department of Corrections, and ACC. It seems as though the response is written up, then it is printed out and the physical copy is signed before being scanned, and finally the scanned copy is sent to the requestor as a PDF. This means that, effectively, the response is delivered in the format of an image of text.

This has obvious accessibility issues for requestors who are blind or have impaired vision, but is also detrimental to sighted requestors.

For example, I recently received such a response from the Department of Corrections which included some long URLs. It is not possible to copy text from these responses without specialised software that many requestors will be unlikely to know much about (OCR – Optical Character Recognition – software can be used convert images of text into actual text). In this case, I had to type the URLs out by hand. Though there were only three of them in this instance, a lengthier response could easily have made this a significant barrier to accessing the released information.

A very long and difficult to type URL.

In the case raised by Nikki Macdonald, I understand she received a PDF response that, unexpectedly, also included large amounts of data. However, because the release was effectively an image of this data rather than a more usable format such as a spreadsheet, it will require a conversion process (either by hand or via OCR software) before it can be used.

I received a similar response in 2015, when I requested documents from the Ministry of Health but did not specify a preferred format. One of the documents I had requested was a large spreadsheet, but it was released to me as a PDF. Though this particular PDF was at least text-based, I still had to use specialised software to convert it to a usable format. I have made the PDF I was sent available online: compliance-of-comp-meds-manufacturers-2-oct-2007.pdf

While this issue can be pre-empted by requesting that the information should be released in a specific accessible format, such as an XSLX or CSV spreadsheet for data, this is not always realistic when it’s unclear whether or not the response is likely to include this type of information.

Despite section 16 of the OIA laying out certain responsibilities here, this approach is also not always respected by the agency. I have received multiple datasets as image-based PDFs despite requested explicitly as XSLX or CSV spreadsheets.

Here is an example of one response where I had to transcribe data by hand. The full response contains more data, as it continues onto the next page:

A table of data.

I have also assisted other requestors in transcribing information received this way, during which I was reminded that it can be quite a painful and time-consuming process for a requestor who is neither a fast typist nor able to use specialised OCR software.

In cases where OCR software is used to convert an image-based PDF response into a text-based one, which allows responses to be searched for certain keywords, this is often hampered by “Released under the Official Information Act” watermarks that are commonly overlaid on each page of released documents. These watermarks interfere with the software’s character recognition, so that portions of the text must instead be transcribed by hand. Poor quality scans, such as misaligned pages, can also cause issues with using OCR software to convert released information into an accessible format.

OCR software can also incorrectly transcribe parts of text, so it is always necessary to compare its output with the original PDF. This can be a time-consuming process, and is inaccessible to requestors who are blind or have impaired vision.

My impression, from talking with other users of the Official Information Act, is that this is a common and frustrating experience, widely regarded by requestors as being unnecessary and obstructive.

I accept that there may be cases where it would be difficult for an agency to release a text-based document with appropriate redactions, and where the reason for each piece of information withheld is clear and the withheld information cannot be accessed. However, there are clearly many cases where information could be released in a more accessible format, and where this would happen if the requestor had known to specify such a format in their request. I would hope that it might be expected that, when information can be released in any of several different formats with similar effort, the most accessible format should be used by default.

For example, after having seen me express frustration on social media about having to transcribe length URLs by hand from an OIA response, the Department of Corrections has agreed that they will now include links in the body of the email sent as a response. Although this only affects a small part of the response, this change would make the URLs significantly more accessible by allowing them to be used directly, without having to transcribe them from the PDF attachment.

I have looked through the Official Information Act and the official information legislation guides on your website for guidance on this accessibility issue.

The closest thing I have found is the relatively recent case note regarding a request from a prisoner who was unable to access information publicly available via the internet, after their request was refused under section 18(d).

Unfortunately, I have not found anything specific to this issue of accessibility. I’d greatly appreciate any advice you might be able to give on making requests in a way that should make a more accessible response more likely.

I would also like to suggest that a guideline on this issue might significantly improve both the experience of making requests for official information and the accessibility of released information.

I think it would be reasonable to expect agencies to release information with an accessibility-first mindset. This would be consistent with standards such as the New Zealand Government Web Toolkit, which recommends the following:

“Choose formats that are both easy to use and most likely to be accessible in the future. Choose openly documented formats over closed formats, or ensure that material released in closed formats is also accompanied by an equivalent in open formats.”

Particularly as many requests are made publicly available, either by the agency, through a service such as FYI.org.nz, or by direct sharing from person to person, it is important to consider issues such as accessibility for the blind and visually impaired for every request.

It is my opinion that accessibility should be considered an important part of information being made “available”, as per the Official Information Act’s Principle of Availability.

Ngā mihi,
Mark Hanna

Asking the government for information

Asking the government for information

You have the right to ask the government for information. Because of a law called the Official Information Act (OIA), they’re obliged to give it to you unless there is a good reason not to.

You’ve likely seen the OIA mentioned in the news. Phrases like “Documents released to [news outlet] under the Official Information Act” can often be found in important news stories. It’s in indispensable tool for holding the government to account.

Some of my articles here have also been based on information that I only had access to because of the OIA. For example, my articles about ACC’s funding of acupuncture have all been based heavily on information released under the OIA.

My articles on strip searches in prisons, organ donation, and the history of the complementary medicines industry have also all used information released under OIA.

An interactive visualisation I did of police use of force data released under the OIA found a place on the Herald Insights website earlier this year, and the story accompanying it was published on the front page. Organ donation statistics I’d requested also made the front page of the Herald in April of 2016.

It’s such a great and flexible tool, so I want to make sure everyone is able to use it.

It can feel a bit daunting if you think you might need to read the legislation itself to understand what you need to do. So to help make the OIA more accessible, I’ve written a guide to its use based on my experience with it.

You can find the guide at oia.nz

OIA Guide

I hope you find it useful!