Beyond Journal Articles: Navigating the NTRO (Non-Traditional Research Outcome)
Siobhán McHugh / University of Wollongong, Australia

WARNING: This article contains names and images of Aboriginal people who have died.

Aboriginal artist Alma Nungarryai Granites is interviewed by Siobhán McHugh for a radio documentary as a Non-Traditional Research Outcome, Yuendemu, Australia, 2016.

I am walking around the remote Aboriginal community of Yuendemu, in the Australian desert, when a pack of fierce dogs appears. I tentatively record their loud barking.

Later, a procession of Aboriginal women emerges from the art centre, their dark bodies painted with white ochre in ceremonial markings. They are talking and laughing. They begin a slow dance, their singing rising and ebbing as I record.

At the art centre, I interview Alma Nungarrayi Granites, a renowned painter. Alma draws her ancient Star Dreaming, a celestial formation. But she’s absorbing Western art thinking too. “Nothing is a mistake in painting,” her friend Gloria told her. “Just work with the mistake: that’s how I learned,” says Alma. Gloria comes from Chile. She is an art conservator, a martial arts enthusiast and an animal lover. She’s organised vaccinations and adoptions to improve the health of the local dogs. “There were about 700 dogs to 1000 people when I came,” she says. She personally looks after about fifty of them.

Gloria Morales (left) and Alma Nungarryai Granites at Warlukurlangu Art Centre, Yuendemu, Australia.

Gloria introduced over 200 colors to the artists, who previously used only red, white, black and yellow ochres from the land. She and her co-manager, Cecilia Alfonso, also from Chile, have ramped up sales from about 300 artworks a year in 2001 to about 8,000 now. Tourists are surprised to hear the artists take the market into account, she tells me. “Are they working for money?” one asked. “I work for money,” Gloria shot back. “Don’t you?” Her voice shows her annoyance. It’s patronising Aboriginal women to suggest they don’t think about earnings.

All of these audio “scenes” will build depth and character for a story I’m telling as part of a research project funded by the Australian Research Council (ARC), Australia’s main academic research body. [1] It’s led by an art historian, Professor Ian McLean, in partnership with Margo Neale, Senior Indigenous Curator at the National Museum of Australia. We’re seeking to document the significant but little known cross-cultural relationships that influence the production of Aboriginal art today—an important economic and cultural activity. This research will first be published, not as a refereed journal article, but as a crafted audio storytelling documentary, The Conquistador, the Warlpiri and the Dog Whisperer, broadcast on national radio (ABC 2018) and as a podcast. [2]

Alma Nungarryai Granites with her painting of the Star Dreaming, or Seven Sisters, at Warlukurlangu Art Centre, 2016.

In the podcast, sound, as well as speech, expresses aspects of the community: the culture of singing and dancing, Gloria’s closeness to the ubiquitous dogs, the affection and respect the two women share, children speaking their Warlpiri language. The holistic audio artifact allows us to appreciate at many levels, including the sensory, the cross-cultural dimensions of Indigenous art production—and in choreographing these sound recordings into a layered, affective, creative work, I am creating not just an engaging and accessible documentary, but a scholarly “non-traditional research output” (NTRO).

CRAFTED AUDIO STORYTELLING as a NTRO

Since 2010, NTROs have been classified by the ARC and audited alongside traditional books, book chapters, journal articles and full conference proceedings in periodic assessments of Australian universities’ Excellence in Research Australia (ERA). The ERA reports provide “a nationwide stocktake of discipline strengths and areas for development” and are a crucial indicator of a university’s standing. I’ve had two NTROs processed by ERA: a two-hour radio documentary/podcast, Marrying Out (ABC 2009), [3] about religious bigotry and interfaith marriage, and a one-hour radio documentary, Eat Pray Mourn: Crime and Punishment in Jakarta, made with an anthropologist, Dr. Jacqui Baker, about extrajudicial police killings in Indonesia (ABC 2013). [4]

To approve a NTRO, ERA applies rigorous standards of peer review—crucial for the evaluation of any academic research. Applicants submit a research statement, which describes the background, contribution, and significance of the particular work. ERA defines research as: “the creation of new knowledge and/or the use of existing knowledge in a new and creative way to generate new concepts, methodologies, inventions and understandings.” NTROs are admissible in six categories: live performance of creative works, original creative works, recorded/rendered creative works (such as my crafted audio works), curated or produced substantial public exhibitions and events, research reports for an external body, and portfolio.

Far from being an easy option, NTROs receive even closer scrutiny than conventional research outputs, as Professor Ross Woodrow (2016) notes.

They have been scrutinised by editorial or peer-review selection processes by publishers, gallery directors, curators, and selection panels before publication. Post-publication, the outputs have undergone verification and evaluation by Research Deans and Officers in each university and, finally, by external ERA reviewers. In addition to this, a number of universities, such as the University of Sydney, also appoint external peer assessors to oversee all creative research outputs collected in its research data repository. [5]

Thus, having my documentaries accepted by the national broadcaster (a competitive process with generally under 20% acceptance rate, similar to major research grants in Australia) and/or winning endorsements such as prestigious awards (e.g. Marrying Out and Eat Pray Mourn won gold and bronze at the New York Radio Festival) constitute tiers of peer review.

I argued to ERA that Marrying Out created new understandings of existing knowledge. For the series, I interviewed 50 people about marrying across the bitter Catholic-Protestant divide that bedevilled Australia, an echo of the troubled colonial history between largely Catholic Ireland and Protestant England that dated back centuries. Those tensions were documented.

What was new in my synthesis was the visceral experience for listeners of sharing the pain of family feuds and societal bigotry: it was carried affectively in the interviewees’ voices and in their non-verbal sighs and tears. [6] It was amplified by specially composed music and historical references powerfully evoked by the archival recordings and actuality I selected—verite recordings of church weddings, funerals, family scenes, royal visits invoking old Empire worship, sectarian taunts performed with vicious gusto by actors and with blithe unconcern by today’s children, for whom they held no currency. This effect on listeners was maximized by the relational way I mixed these sounds in order to heighten their affective power: e.g. having the taunts float over an ethereal boy soprano singing a Catholic hymn, in an emotive evocation of the conflict between prejudice and spirituality. Finally, the original oral history interviews were archived at the National Library of Australia, open to scrutiny and further research.

After Julia O’ Brien, a Catholic, married Errol White, a Protestant, in Sydney in the 1940s, she was ostracised from her family and barred from her father’s deathbed. Their story features in Marrying Out.

In a similar way, in Eat Pray Mourn, Baker and I argued to ERA that hearing the personal responses of a wife, a mother and a sister to the deaths of their loved ones, shot summarily by police, could give listeners a deeper, felt understanding of the extrajudicial killings than would a lengthy journal article. Once I found myself defending the finer points of audio narrative craft to the university’s Ethics Committee (IRB), who wanted us to protectively anonymise the crusading sister of Yusli, a young man killed on a trumped-up charge of motorbike theft. When we first meet Yusli’s mother, she rattles off the names of her six children, including his sister, Yeni. “Everyone is Y,” the mother says with a laugh. Yeni adds: “We gave Yusli five letters, more than the others…more posh. We never thought he was fated to die.”

If we were to remove the names of Yusli and Yeni, we would have to lose that poignant scene. Stories depend on character and voice as well as plot, I told the sceptical social science academics on the committee. And this scene is setting up Yusli’s mother and sister as characters we care about: it’s crucial to getting listeners to engage with the documentary’s underlying purpose, of examining police behavior. Further, far from endangering Yeni, using her name affords her the protection of Western media attention. In the end, the committee approved the original segment.

Yusli’s family and friends mourn at his graveside.

NEW DIRECTIONS for NTROs

Australia was an early adopter of NTROs; my own University of Wollongong graduated the first practice-based Doctor of Creative Arts in Australia, in Visual Arts, in 1988. Practice-based and practice-led PhDs are now increasingly common in the humanities. Australian scholar Mia Lindgren has examined how radio journalism offers a model, but they are also common in creative writing, film and media studies, visual arts and theatre.

Some scholars use podcasts not as NTROs but as a way of increasing their non-scholarly engagement with the broader community: US philosopher Professor Barry Lam was an early adopter, with Hi-Phi Nation, “a show about philosophy that turns ideas into stories,” while Australian historian Dr. Tamson Pietsch hosts the popular History Lab podcast, whose tagline, “Australia’s only investigative history podcast,” indicates its role in examining the historical process as well as showcasing new understandings of history.

The episode itself does not represent new research…it is more a communication or interpretation (by the producer) of existing work. …Our commitment is to doing the work of thinking and making meaning, not for our listeners, but with them. [7]

History Lab is a finalist in the 2019 Australian Podcast Awards in the documentary/storytelling category, a testament to the collaboration of its academic hosts with their university radio station, 2-SER. Academics increasingly seek out skilled audio producers to co-create conversational podcasts on academic themes: Stuart Hall: In Conversation, produced by KUT journalist Rebecca McInroy and hosted by University of Texas sociology professor, Ben Carrington, celebrates the life and achievements of the late cultural studies theorist.

A Canadian scholar is testing the podcast NTRO further, as a form of non-traditional scholarly publishing. Hannah McGregor, an Assistant Professor of Publishing at Simon Fraser University in Vancouver, is “in the midst of a collaborative research project with Wilfrid Laurier University Press in which we’ve subjected my podcast, Secret Feminist Agenda, to peer review.” [8]

In Secret Feminist Agenda, McGregor interviews a broad range of feminists and reflects on “the insidious, nefarious, insurgent, and mundane ways we enact our feminism in our daily lives.” Four lengthy peer reviews, by digital humanities, social justice and literature scholars, analyse the first two seasons and are published online. The reviews provide a valuable assessment of many aspects of the podcast, but neglect to appraise the use of an audio format: a bit like having a review of a journal article fail to address the clarity, correctness and style of the writing—an integral aspect of its ability to communicate research. If podcasts are to be put forward as research outputs, they need to be evaluated by someone who is also audio-literate.

The potential of the podcast medium to deliver innovative research opportunities is being harnessed in highly imaginative ways. In the UK, for instance, the BBC has teamed up with three universities to develop an absorbing audio “eco-thriller” or sci-fi story, Forest 404, which incorporates sounds of the natural world as plot elements. These sounds are developed as accompanying tracks, along with short talks by a PhD researcher, who uses listener feedback to study how natural sounds can impact mental health. I look forward to seeing many more interdisciplinary research collaborations that tap into the awesome power of audio and the new medium of podcasting in fresh and exciting ways.

Image Credits:

1. Aboriginal artist Alma Nungarryai Granites is interviewed by Siobhán McHugh for a radio documentary as a Non-Traditional Research Outcome, Yuendemu, Australia, 2016 (Author’s personal collection).
2. Gloria Morales (left) and Alma Nungarryai Granites at Warlukurlangu Art Centre, Yuendemu, Australia (Author’s personal collection).
3. Alma Nungarryai Granites with her painting of the Star Dreaming, or Seven Sisters, at Warlukurlangu Art Centre, 2016 (Author’s personal collection).
4. After Julia O’ Brien, a Catholic, married Errol White, a Protestant, in Sydney in the 1940s, she was ostracised from her family and barred from her father’s deathbed. Their story features in Marrying Out (Susan Timmins).
5. Yusli’s family and friends mourn at his graveside (Jacqui Baker).

Please feel free to comment.

1. McHugh, S, McLean, I, Neale, M (2018), Heart of Artness podcast series Season One: Five episodes, viewed at http://artness.net.au/about/ [↩]
2. McHugh, S (2018). The Conquistador, the Warlpiri and the Dog Whisperer. Radio documentary, Australian Broadcasting Corporation, Sydney, 18 May 2018, 55mins Viewed at https://www.abc.net.au/radionational/programs/earshot/the-conquistador,-the-warlpiri-and-the-dog-whisperer/9617950 [↩]
3. McHugh S, (2009). Marrying Out: Part One – Not in Front of the Altar. Radio documentary, Australian Broadcasting Corporation, Sydney, October 2009, 55mins Viewed at https://www.abc.net.au/radionational/programs/archived/hindsight/marrying-out—-part-one-not-in-front-of-the-altar/3068558 [↩]
4. Baker, J and McHugh S, (2013). Eat Pray Mourn: Crime and Punishment in Jakarta. Radio documentary, Australian Broadcasting Corporation, Sydney, April 2013, 55mins
   Viewed at https://www.abc.net.au/radionational/programs/360/eat-pray-mourn/4598026 [↩]
5. Woodrow, R (2016), “NTRO: A Model for Change,” NITRO, August 11 2016. Viewed at https://nitro.edu.au/articles/edition-2/ntro-a-model-for-change?rq=Woodrow [↩]
6. See McHugh, S. A. “The affective power of sound: oral history on radio.” The Oral History Review 39, 2 (2012): 187-206. Viewed at https://ro.uow.edu.au/creartspapers/345/. NOTE: it is essential to listen to the audio clips in conjunction with reading the article. Audio at https://drive.google.com/drive/folders/1jesv_GhLcpVrdi4rwDet4SGM [↩]
7. Pietsch, T (2019), personal communication to the author, 5 February 2019. [↩]
8. McGregor, H (2019), personal communication to the author, 16 February 2019 [↩]