A research project: Gathering apartment data for the government

 

Summary

Here’s a question: How many individual apartments exist in Sydney?

The government would probably know that, right?

Well, turns out they don’t. There isn’t great data available about apartment complexes (known as strata plans). So Service NSW, the service arm of the NSW state government, planned to release a simple web form that each strata plan should fill out every year. It would let the government gather accurate, up-to-date information about apartment buildings and complexes.

Sounds simple enough, right?

I was brought in to run validation research on the MVP and design for the future state.

Turns out it’s not that simple…

 
 



Over 8 weeks, I ran 3 rounds of user interviews, each round with a different persona, each time speaking to 5-10 users.

The basic user research process:

  • Identify risky assumptions we’re making as a team

  • Come up with clear learning goals (as a team)

  • Be clear on the persona we need to speak to

  • Write a screener to identify interviewees

  • Find them (or let someone else find them 😜)

  • Write moderator guide

  • Create whatever we want to test - prototype, experiments, card sort, something creative…

  • Run user interviews

  • Have someone from the team take notes in the interview

  • After the interviews, synthesise

  • Agree on what insights we learned

  • Agree on actions to take

Before the research

  • The goal was to release a very simple form, to be filled out annually by every strata committee (aka every apartment building), entering basic information about the property.

  • It was assumed that for security, to fill out the form, you’d need to log in. After filling the form in, a “strata profile” would be created, making the data public. The profile would be connected to the account of the person who filled in the form.

  • Anyone can create a Service NSW account, without proving their identity. It’s just authenticated with email address and password.




 
illustration of a pair of glasses, metaphorically indicating "shadiness"
 
 
 

An unusual research situation…

At one point in the research, I was feeling like my user interviews were going in strange and sometimes hostile directions. What was going on?

Well… our user research had been infiltrated by political lobbyists.

I had to iterate my interview questions to subtly but clearly dig into the potential biases each interviewee might have. We also had to very quickly iterate on where we were finding our interviewees :)

 
 

Remote research in the time of covid. We used Miro for everything…

Personas

A persona should be a living document, changing with every new learning. For that reason, I deliberately keep them lo-fi and looking like wireframes or drafts, so the team doesn’t hesitate to edit them as often as needed.

Proto persona before round 1 vs. persona after 2 rounds of research



Key findings from the research

We had the user all wrong!

We were incorrect about who we assumed the main user of the form would be. Rather than a single committee member who’d fill the form out once, the main use case was in fact professional strata managers, who might need to fill out the details up to 1200 times a year for different properties.

It could be a different person every year filling this form out.

Strata committees almost universally do not have a shared email address or anything that could be used as a shared login.

Forcing users to log in in the way originally designed was therefore likely to cause an unintended consequence - the data would be accurate for the first year, then by the second year it would be impossible for many strata committees to log in to update it. The data would become unreliable extremely fast. This could have caused an expensive embarrassment to the government.


The “KTM” - Known TroubleMaker…

Throw a bunch of unrelated people together who each own part of a building, and ask them to make decisions about it together. Now add in the fact that it’s probably each of their life savings and biggest investment. Not a recipe for smooth sailing.

Several interviewees told us about some intense situations ranging from annoying neighbour to restraining order.

We really couldn’t let just anyone update these details - aside from it being important the data was accurate, it could also put real people at risk.

Presenting the findings

For every round of research, I worked with the Product Manager to put together a presentation to present back the findings to the stakeholders and those we needed to convince.


 
 



Key outcomes

Bulk upload
Firstly, because we now knew our key users were not individuals but strata managing agents, a new feature was built to allow managers of multiple properties to bulk upload, rather than having to fill in the form manually dozens or hundreds of times. This meant the initial data set had a much higher chance of being completed.

No more broken ownership model
The previous design assumed that the first user to fill out the form therefore became owner of the “strata profile”. A whole new IA was designed, so that instead there would be a public strata profile without one owner, which could be updated by any logged-in user who went through an identity check. In this way, the data could still be edited by anyone, but having to upload your ID should be a very strong disincentive to lying to the government.

The new model - log in, but also prove your identity.

 

What I learned

  • It’s so different designing products for government. There can much less focus on providing user value within the team. At one time, I asked, What will make people use this if it’s not obviously providing them value?
    The answerwe passed legislation. They have to fill it out by law...
    Luckily Service NSW is super focused on user value.

  • Similarly, when your user will be forced by law to use your product, accessibility can not be an afterthought.

  • Working in politics - a lot of decisions were made in a way because of politics. For example - a senator really liked this “business profile” product that had previously been built, and wanted it to be repurposed as a “strata profile”. It was totally ill-suited and 99% of people could see that. We put a lot of work into gathering data to convince the right people of that.

  • I’ve been fully on board with the concept of shipping early and often, releasing an MVP you’re embarrassed of, and iterating after your product is already in the wild. But for government, these concepts are only partly applicable. If there will be news stories about your new product, you don’t want to release it obviously half-baked. The bar for “Minimum” in that kind of MVP just has to be pretty high.