Contributor Feedback on Usability in Open Source Software
Wang, Wenting, Jinghui Cheng, and Jin L.C. Guo. 2020. “How Do Open Source Software Contributors Perceive and Address Usability? Valued Factors, Practices, and Challenges.” IEEE Software. IEEE Computer Society. doi:10.1109/MS.2020.3009514.
According to Wikipedia, usability is the degree to which a software can be used by specified consumers to achieve quantified objectives with effectiveness, efficiency, and satisfaction in a quantified context of use. For a Open Source Software, this becomes a community opinion. Since the advent of collaborative tools like GitHub which help the software development progress at a faster pace, usability of the developed tools becomes a cause of concern. This paper presents a community view by analyzing 84 survey responses.
Abstract
The major pattern observed in the survey is that developers often adopt software-centric approach to design the product (focus on the software stack, ease of development) rather than user-centric approach. While they recognize the importance of usability, most of the developers find it difficult to consolidate and understand the community needs.
Introduction and Background
Open Source Software development has challenges like geographically distributed communities, diverse contributor background, absense of hierarchy in members and possible asynchronous communication. This can lead to the development process becoming localised, while the software can be used globally.
While usability is considered important, it was only treated as a feature of the software rather than making it the center around which the software is built. However, considering the tremendous growth of the open source world, it becomes cardinal to understand the community standpoint on the issue.
The key questions answered in the survey, paraphrasing from the paper, are
How do OSS contributors perceive the determining factors associated with usability?
What are the prominent approaches OSS contributors currently carry out to address usability issues?
What are the prominent barriers and difficulties they faced with respect to addressing usability?
Survey Methods
The survey demographics was varied between software engineers, managers, designers, software architects, data scientists and others. Their contributions were wide ranging, from feature development and code maintainance to UI design, public relation and release management.
However, researchers mention that that the demographics was only that of GitHub users. This can possibly skew the data, as any activity on this platform will be affected by the features of the platform itself. However, it goes without saying that any prominent issue that is captured on this platform can represent a non trivial subset, if not most of the issues about OSS usability.
Survey Results
Results from the survey are divided into three categories - Valued Factors, Practises and Challenges.
Valued Factors
Researchers identified six themes that are valued by the software practitioners. These are listed down in the decreasing order of the popularity.
Usability related System Characteristics Participants value efficiency, consistency and customizability. The researchers mention that people involved in the development seemed to have followed the usability guidelines for achieving these system centric qualities. This is quite possible since these qualities would be paramount to the adoption of software.
Other System Characteristics System characteristics such as reliability, maintainability and feature completeness conrtibute to usability as well. Code quality affects usability to some extent, as it becomes easier to iterate over the software and make it better.
User Support The interaction of development team with the users of the software; to respond to the issues, and feature development on request. This improves the trust among users in the software, that their concerns are heard and addressed.
Environment and Market Audience size, competitor performance and software impact over users and society. Higher the popularity of the software, more users use it resulting in more bugs or feature requests to handle varied scenarios. Competitor performance and impact ensures that there will be a higher effort in the development process and more iterations. These further improve the usability.
User Characteristics Understanding the background of the target users can aid the usability design as it will make sure that the contributors plan the way the software is to be used - either focussing on the power users or make the software diverse.
Resource and Mindset Financial support, time, and expertise of the members involved contribute to the usability of the OSS as well.
Practises
Researchers identify the methodology followed by the practitioners in their usability design for the OSS.
Explore Needs Participants welcome the feedback on usability. They three types of people who provide these are the general users (who provide the feedback in ITSs), power users involved in the the development process and developers themselves by discussing with the users. Sometimes the usability features are also inspired from research literature, competitor systems and opinions by consultants.
Approach Design There are three approaches that are used by the participants.
An interactive trial and error process was followed by majority of participants where a feature is developed, then the feedback for the usability is captured using ITSs and further itrations to the software is made based on the requirement.
A participatory approach where the users are directly involved in the development.
Designing a system based on their own opinions, which are influenced from guidelines or tutorials.Evaluator Usability The usability evaluation is done mainly through ITSs, if done at all. One more approach that is followed is an informal approach where is the testing is done by some personal connections, power users or anyone who is closely associated with the software and not the end users. Sometimes, the usability is evaulated by UX students, and has resulted in success. Apart from these, the usability can be evaluated by the internal evaulation of development team (more like developer testing) which follows the line of thinking that developers are also users.
Challenges
Challenges faced by the participants towards usability of the OSS is as follows.
Mindset The developers tend to be system-centered while designing a system and focus on improving the feature set and performance and neglect usability. This can be tackled by having a user feedback phase in the development cycle, and thereby nudging the developers towards paying attention to usability.
Understanding User Needs User needs evolve frequently, and are also difficult to map the feedback to development tasks. This can be improved by having better discussions with the users; many times, a complicated feature can be avoided by just making minor changes to the existing code to address the underlying issue. Better software to gather feedback data also would help in addressing the issue.
Addressing User Diversity Since OSSs have a diverse user base, the feature requests need to prioritized. Addressing usability for a large user base is hard; some power users might need customizable features, but some users might feel this can add unnecessary complexity to their workflow. Finding a balance between these two can enable easier and better adoption.
Development Issues Framework, library limitations, project complexity and lack of transparency in development process can affect usability as well. Over the course of years, new developers can come in and some may move out of the project, resulting in different standpoints and ways of approaching usability. This can be addressed by the better onboarding practices for developers.
Discussions and Future Work
Supporting a Deeper Engagement with a Diverse User Base The research clearly indicates that developers working on user feedbacks have the best chance of achieving good usability. Main problems around this are diverse and sometimes contradictory feedbacks, and not having enough demographic representation. Since not a lot of work is done in this field this is a possible research direction. Having features that generate use cases from user comments in ITS could be the way to go.
Promoting Motivation for Usability at the Community Level One interesting finding, as pointed out by the authors is that organization-supported OSS projects thrive due to an active feedback cycle; volunteer driven projects struggle at this. Having usability related tags for both developers and for users to provide feedback in ITSs can help promote a user feedback cycle in OSS.
Enhancing Tools and Techniques for OSS-Oriented User-Centered Design that integrate user response in development process can aid the developers massively. Static analytic tools can detect usability related issues in the code and help resolve challenges around certain challenges like inconsistent use of UI libraries.
Thoughts
This survey acutely captures the thoughts of the developers regarding usability while developing a software. Any software should be designed around usability. Easier and intuitive a software is to use, easier it is to improve the adoption. The researchers offer valuable suggestions improve the mindset in developers.
The results can also be pivoted in such a way, to understand how the valued factors, practises and challenges influence one another. For example, the participants that value certain themes and follow certain practises can face a specific set of challenges. This model can further be used to course correct and avoid such pitfalls.
I believe that this paper is relevant to developers, to inculcate a usability-centric mindset. It also points to a research and development direction for ITSs in terms of having an easier user feedback loops (take note, GitHub!).