The Australian Government Department of Education, Skills and Employment (DESE) has introduced a number of new reporting requirements for all Higher Education Providers (HEPs) to improve monitoring of the research training system. Research end-user engagement is now included in the required data that all universities must collect and report to DESE and so ANU Higher Degree Research (HDR) Candidates are now required to report on their own engagement with research end-users. The ANU is collecting this information through collection rounds facilitated by a Quatrics online form.
These changes are in response to the Australian Council of Learned Academies (ACOLA) Review of Australia's Research Training System, and also support the new Research Block Grant funding arrangements for universities. The ACOLA review found that there was a strong need to support and increase engagement between the research training taking place within universities, and research-end users such as governments, businesses, and communities.
DESE has provided a set of definitions for different types of external engagement activities. There are five components that must be reported on:
- Research internships with a research end-user
- Joint supervision by a research end-user
- Jointly funded or fully funded by a research end-user
- Formal training on industry engagement
- Other commercialisation and engagement activities.
What is a 'research end-user'?
A research end-user is defined as an individual, community or organisation outside of academia that will directly use or benefit from the output, outcome or results of the research. This may include businesses, governments, non-governmental organisations (NGOs), communities and community organisations. Research end-users are not:
- Other HEPs (universities)
- Organisations that are affiliates, controlled entities or subsidiaries (such as Research Institutes) of a HEP
- Equivalents (international or domestic) of the above exclusions.
Research internship is defined as a temporary paid or unpaid position (of at least 30 days) with a government, business, community, or organisation outside of academia where a student has undertaken research and development related to their core research program. Research visits to other universities or institutes are not considered research internships. Examples of a Research internship are below.
Jointly funded or fully funded by a research end-user
Jointly funded or fully funded by a research end-user is where a research end-user contributes financially to the cost of an HDR student's course of study or to other costs such as a stipend for general living costs. The financial contribution must be awarded specifically to an individual HDR student rather than as part of funding for a general research project that a student may be involved with. Examples of joint/fully funded by research end-user are below.
Formal training on end-user engagement
Formal training on end-user engagement is defined as any formal training recognised by ANU that focusses on preparing a student for work with a research end-user. This includes intellectual property, management/leadership, collaboration, entrepreneurship, and research commercialisation. Training can be administered by a research end-user organisation or by ANU. Examples of formal training on end-user engagement are below.
Other commercialisation and engagement activities
Other commercialisation and engagement activities is defined as an arrangement with a research end-user that enables experiential learning related to the student's HDR. This includes practicums or performances, Research and Development (R&D) consultancy work, R&D commercialisation work, entrepreneurship, community engagement/outreach, and research extension work either with or for a research end-user. These activities can be paid or unpaid, and no minimum amount of learning days applies. The activities exclude research internships unless they are shorter than 30 days, and exclude joint funding arrangements. Examples of other commercialisation and engagement activities are below.
Collection of Data
For further information and full DESE definitions, see the DESE's 593 Higher degree by research end-user engagement code webpage (definitions can be found through Glossary links on that page).
Collection 1 - 2018: Census period 1 January 2018 to 31 March 2018
This was the first data collection for the HDR research end-user indicator and was collected by individual ANU Colleges and Schools in February to March 2018 through a combination of forms (PDF and Qualtrics) facilitated by the ANU Planning & Performance Measurement (PPM) Division on behalf of the Dean HDR. Indicator data was reported to the DESE on 31 May 2018.
Collection 2 - 2018: Census period 1 April 2018 to 31 August 2018
The second data collection round utilised a central Qualtrics online form facilitated by PPM Division on behalf of the Dean HDR. Access to the form was via an email link. The email was sent to all current HDR Candidates on 1 August 2018 from ANU_ODHDR firstname.lastname@example.org. The form was available to be completed online until 7 September 2018. All HDR Candidates were required to complete the form. Indicator data was reported to the DESE on 31 October 2018.
Collection 1 - 2019: Census period 1 September 2018 to 31 March 2019
Collection 1 - 2019 utilised a central Qualtrics online form facilitated by the Graduate Research Office (GRO) on behalf of the Dean HDR. An invitation email was sent to all HDR Candidates on the 25 March 2019 from ANU_GRO email@example.com. The form was available to be completed online until 15 April 2019. All HDR Candidates were required to complete the form. Indicator data was reported to DESE on 31 May 2019.
Collection 2 - 2019: Census period 1 April 2019 to 31 August 2019
Collection 2 - 2019 utilised a central Qualtrics online form facilitated by the GRO on behalf of the Dean HDR. An invitation email was sent to all HDR Candidates on the 5 August 2019 from ANU_GRO firstname.lastname@example.org. The form was available to be completed online until 6 September 2019. All HDR Candidates were required to complete the form. Indicator data was reported to DESE on 31 October 2019.
Collection 1 - 2020: Census period 1 September 2019 to 31 March 2020
Collection 1 - 2020 utilised a central Qualtrics online form facilitated by the GRO on behalf of the Dean HDR. An invitation email was sent to all HDR Candidates on the 16 March 2020 from ANU Graduate Research Office email@example.com. The form was available to be completed online until 17 April 2020. All HDR Candidates were required to complete the form. Indicator data was reported to DESE on 31 May 2020.
Collection 2 - 2020: Census period 1 April 2020 to 31 December 2020
Collection 2 - 2020 utilised a central Qualtrics online form facilitated by the GRO on behalf of the Dean HDR. An invitation email was sent to all HDR Candidates on the 10 August 2020 from ANU Graduate Research Office firstname.lastname@example.org. The form was available to be completed online until 11 September 2020. All HDR Candidates were required to complete the form. Indicator data was reported to DESE on 31 October 2020.
Collection 1 - 2021: Census period 1 January 2021 to 30 June 2021
The current collection round utilises a central Qualtrics online form facilitated by GRO on behalf of the Dean HDR. An invitation email will be sent to all HDR Candidates on the 7 June 2021 from ANU Graduate Research Office email@example.com. If you cannot see the email in your ANU email inbox on 7 June 2021, please check your junk or spam folder. The form will be available to complete online until close of business on 2 July 2021. Further details on this collection round are contained within the FAQs section below. Indicator data will be reported to DESE on 2 August 2021.
The following are examples of research internships with a research end-user, as long as the period is > 30 days (and the internship is either paid or unpaid):
- a Parliamentary Fellowship Program (organised in a joint partnership with an international NGO)
- a secondment to an Australian Governement Office to assist with creation of an intergenerational dataset
- Working in the Parliamentary Library (on a Summer research scholarship)
- Working in the Research Centre of an NGO.
Examples of where you would answer 'Yes' to the question on Funding support and enter the details in the form:
- living stipend and sponsorship for training and conference attendance respectively (to suppport research) are provided by an NGO Research centre
- top-up HDR stipend scholarship from a government organisation such as the Grains Research and Development Corporation.
Formal training on end-user engagement
Examples of where you would answer 'Yes' to the question on Formal training and enter details in the form:
- completed an ANU course to meet HDR coursework requirements on the topic of research commercialisation.
- completed an external training course or workshop on management and leadership and reported this in my annual plan and report milestone
- attended an ANU seminar or training series facilitated by my school or department on intellectual property and entrepreneurship.
Other commercialisation and engagement activities
- a research publication or book produced from collaboration with a research end-user, as long as the engagement meets requirements for one of the activities above
- a paid or unpaid position (i.e. research internship) under 30 days in duration (see also the Research internships examples, above).
Frequently Asked Questions (FAQs)
Q: Who should complete this research end-user reporting requirement?
A: All current ANU HDR Candidates. Note 'current' means HDR Candidates within a HDR program at any point in time between 1 January 2021 to 30 June 2021. This includes HDR Candidates who are enrolled, on leave for part of this collection period, under examination, undertaking corrections, and awaiting approval or conferral of a HDR award.
Q: Should I complete the form even if I haven't undertaken any research end-user engagement during my program or in this reporting period?
A: Yes you should still complete the form and confirm that you have not undertaken any activities. Once you have answered a few quick questions, you will then be directed to the declaration page for your sign off.
Q: Who will have access to the information submitted and where will it be stored?
Q: What should be completed by HDR Candidates?
A: An online form with questions relating to your engagement with research end-users.
Q What connections do I need to access and complete the form?
A: You will need an Internet connection at all times whilst filling in the form as the software (i.e. Qualtrics) used to deploy the form is web based.
Q: What operating systems are compatible with the form?
A: The software used to deploy the form (i.e Qualtrics) is compatible with all operating systems.
Q: What devices can be used to complete the form?
A: The form can be filled in using a desktop computer, laptop, mobile phone or tablet device with an Internet connection.
Q: What is the recommended Web browser to complete the form in?
A: To ensure the forms functionality remains operational please ensure that you only use one of the following browsers: Google Chrome (preferred), Safari, or Firefox. Please do not complete the form utilising Internet Explorer or Microsoft Edge as some functionality within the Qualtrics form is not compatible with those two browsers.
Q: What is a general example of research end user engagement?
A: An experience or activity where you have undertaken a paid or unpaid placement, internship, fieldwork, training or joint-supervision arrangement where your research will benefit a research end-user organisation via intellectual property or commercial gain.
Q: What kind of information should I fill in?
A: Some examples are: 1. The dates and details of the organisation where you have done a paid or unpaid research internship. 2. The training you have completed including the title, what it was on, who conducted the training and the date or period it was held. 3. The name and ID number of any supervisors who are located in a research end-user organisation and the details of the organisation.
Q: What if I did not receive the email with the link to the form?
A: First check emails you received on 7 June 2021 and then check spam or junk folders. If you still cannot locate the email please contact the support team via firstname.lastname@example.org, remembering to include your student ID number. Do not use a link that was sent to someone else as the links are unique and tied to each individual.
Q: What information should I have ready before I start to complete the form?
A: Before completing the form you should think about any research end-user engagement activities you have completed during the reporting period (in this case it is the period 1 January 2021 to 30 June 2021) e.g. have you engaged in an internship with a research end-user organisation? If so have ready at hand the dates of the internship, the name and contact details of the organisation. If you have a supervisor on your panel that is located within a research end-user organisation then have ready the supervisor's University ID number, name and contact details of the organisation. If you have completed formal training or other commercialisation or engagement activities then likewise have on hand the details such as the name of the training course or description of the commercialisation or engagement activity, the organisation name and contact details and the dates that it was undertaken.
Q: What if I don't know an answer to one or more of the questions in the form? What if I get to the declaration page but I am not ready to submit?
A: The form will automatically save your responses as you progress through the form. You can move forwards and backwards throughout the form and come back to a question at a later point in time if you do not know the answer at the time. You can then take the time you need to seek help or advice whether it be from this web page, from the help team, from you supervisory panel or from your College or School (such as your HDR administrator, or HDR convenor).
Q: What if the engagement activity is one that I have already reported in previous research end-user engagement data collection such as joint funding that has not changed?
A: Regardless of any information you may have provided in the previous collection round/s you will need to complete this information in the current reporting period.
Q: What engagement and involvement will there be with HDR Candidates supervisors? Will supervisors be able to participate in the End User data collection process?
A: Currently the data collection is targeted at HDR Candidates. If you are unsure if one of your panel members is located in a research end-user organisation then you should contact them to find out. HDR supervisors will not directly participate in the collection or complete the form, however they will be able to provide input through their HDR Candidates as requested.
Q: What if I have more than four engagement activities to report from the period 1 January 2021 to 30 June 2021?
A: The form is limited to capturing up to four engagement activities. If you have more then please email the details through to email@example.com remembering to include your name and student ID number.
Q: What does E593 mean?
A: It is the code used by DESE to record data provided by HEPs on HDR end-user engagement.
Q: How should I complete this requirement?
A: Click on the link provided in the email of the 7 June 2021. Please note when opening the link please ensure you use one of the preferred internet browsers (Google Chrome, Safari or Firefox). It will open up a Qualtrics form where you will be asked some questions. The form is dynamic so that the questions will change depending on your initial answer. If you have not undertaken any engagement activities it will take you directly to the end of the form. If you have one or more engagement activities to report it will enable you to enter the details before submitting. Note that you must submit the form by close of business on 2 July 2021.
Q: How do I go back to the previous page within the form?
A: Within the Qualtrics form there will be a Forward button for you to progress through the form and a Back button to go back to the previous question to edit your answers. Please note that once you click on the Forward button on the 'Declaration' screen your form will be completed/submitted and thus you will be unable to go back to previous screens to modify your answers.
Q: How do I save my progress within the form?
A: The form automatically saves your progress upon clicking the Forward button. If you plan on exiting the form or returning to it at a later time please remember to click the Forward button to save the screen you are on.
Q: How do I return to a partially completed (or incomplete) form?
A: You can return to your partially completed form via one of the links provided to you via email in the initial invitation email sent to you on 7 June 2021 or via any of the reminder emails after 7 June 2021. After clicking on a relevant link you will be returned to the screen that you were last on within the form. Note you will only be able to return to the form during the form opening period (from 7 June 2021 to 2 July 2021).
Q: How do I change answers that I have already inputted to the form?
You can change your answers at any time during the form opening period (from 7 June 2021 to 2 July 2021) so long as you have not already submitted the form. To change your answers just click on the Back button to move back through the form and then you can select different options or update your answers.
Q: How can I ensure that I have completed or submitted the form when there is no "submit" button?
A: Once you have completed the questions on the declaration page, all you need to do is to click on the Forward button to submit the form. A final screen will appear that confirms you have completed and submitted the form. Note this final page also includes a summary of your responses, and a link to download those responses as a PDF.
Q: How will future research HDR Candidates be advised of the End-User Engagement data collection? Will it form part of the centralised HDR induction process?
A: We hope to incorporate information on the collection in future induction resources and on HDR webpages. Note the aim is to eventually imbed the collection of this information into existing business processes where ever possible and new business processes for the remaining elements.
Q: When should I do this?
A: Anytime between 7 June 2021 to 2 July 2021 (inclusive).
Q: How frequent is the HDR research end-user engagement collection undertaken and when is this data reported back to the DESE?
A: Each HEP is required to report this data on a quarterly basis to the DESE. Note: ANU is currently collecting the information biannually and reporting quarterly. The process for collecting the information is currently under review and development with the aim of eventually imbedding the data collection within existing or new business processes.
Q: Where can I access the form?
A: You can only access the form through the email link that was sent to you individually from our system (through the initial invitation email, or through any of the reminder emails). Please do not forward your link to someone else or access the form through a link that another person may have sent to you as the links are unique, and linked to you as an individual.
Q: Why do I need to provide this information?
A: The DESE from 2018 has introduced additional reporting elements for all Australian HEPs to report on their HDR Candidates Research End User engagement each census period. As this is mandatory for the ANU to report on to the Government, it follows that HDR Candidates should provide the information on their engagement to the University (see the HEIMS website).
Q: Why is the Australian Government collecting this information?
A: The new HDR indicators, including the research end-user engagement indicators, are part of a wider plan to improve student data collections to meet recommendations made to Government. The improved data will be used to improve monitoring of the research training system, to inform national research training funding policy and to provide a robust national data set for universities to improve their research training programs. New indicators are particularly important to support monitoring and evaluation of the recent program changes including the impact of recent changes on incentives for universities.
On 7 December 2015, the Australian Government announced new Research Block Grant (RBG) funding arrangements for HEPs as part of the National Innovation and Science Agenda (NISA). These arrangements combined existing RBGs into two programs, the Research Support Program (RSP) and the Research Training Program (RTP). These policy innovations were in direct response to the Review of Research Policy and Funding Arrangements, led by Dr Ian Watt (Watt Review) and are designed to drive greater research-industry collaboration by increasing incentives for success in industry and other end-user engagement. In addition, improvements to program guidelines were made in response to recommendations of the Review of Australia's Research Training System undertaken by the ACOLA Review.
The need for better data and information on Australia's HDR training system itself was a common thread across the different areas of the ACOLA Review. Poor data on the performance of the HDR training system makes it difficult to understand what return is generated from Australian Government investment of more than $1 billion annually and how best to go about improving the system.
In the RBG consultation paper on new block grant arrangements released in May 2016, the DESE flagged its intention to improve income and student data collections to meet recommendations of both the Watt and ACOLA reviews. DESE released a further consultation paper on new research block grant data requirements in January 2017. This paper sought feedback from the higher education sector on detailed implementation options for each of the proposed changes. DESE received strong support for the new HDR research end-user engagement indicator through both of these consultation processes.
Q: Why is ANU collecting this information?
A: ANU is collecting the information for the main purpose of reporting the information to DESE. In addition to mandatory reporting purposes, we are also collecting additional information on engagement activities to ensure that the data collection is as useful as possible for the University. We plan to also use the information for internal reporting purposes, for statistical purposes, to monitor the depth and breadth of our research end-user engagement, to inform committees and working parties on Industry engagement, to inform the development of policies, to enable further engagement with research end-users, to improve administrative procedures, and other purposes that are still yet to be identified or determined.
Research end user definition?
Q: My research involves NGOs overseas, do I need to include this information?
A: Yes, if your research requires the involvement of an organisation that can be categorised as a business, government, non-governmental organisation, community and community organisation, regardless of whether they are in Australia or overseas, this information should be included. However, if the organisation is a HEP, or an affiliate, controlled entity or subsidiary of a HEP, such as a Medical Research Institute, this organisation is not considered a research end-user, therefore you do not need to include this information.
Q: Does a research internship require a formal agreement between the student and the research end-user?
A: Yes, a research internship would generally require a formal agreement between a student and research end-user in order to justify that the student was engaged in a 'temporary position' at the research end-user organisation.
Q: Does the position or agreement need to be named or called an internship?
A: It does not need to be called an 'internship', it just needs to comply with the definition.
Q: How do I determine if my volunteer employment, paid employment or other informal work arrangement with a research end-user should be recorded as an Internship activity?
A: It could be any of these arrangements so long as it complies with the definition.
Q: I receive funding from an external organisation in the form of a research grant or stipend scholarship to cover some of my research expenses, would this be considered a research end-user activity?
A: If you receive funding from an organisation external to the ANU (or other tertiary institution or affiliates/controlled entities), which is specifically used to cover research expenses or living expenses whilst you are a HDR Candidate, you will need answer yes to the question on Funding support and enter the details in the form.
Q: I receive an RTP stipend funded by the Australian Government. Am I jointly funded by a research end-user?
A: No - as the Australian Government is not directly benefitting from your research.
Q: I receive funding as part of an externally-funded (i.e. ARC, ARENA, NHMCR) project. Is this considered jointly or fully funded by a research end-user?
A: If the funding is awarded specifically to you as an HDR student rather than to a general research project that you are involved with, and the organisation meets the definition of a research end-user organisation, then the answer is yes, if not then no.
Q: What does 'formal' training mean? Does formal training require there to be a certificate or award? Can this include coursework undertaken during the HDR program such as a course to assist HDR Candidates on commercialisation? Is it limited to nationally accredited programs or courses? Can it include training such as seminars or workshops?
A: 'Formal' means that the training must be endorsed by the University. This need not result in a certificate or award and is not limited to nationally accredited programs or courses. It could include coursework undertaken during a HDR program and training such as seminars and workshops as long as it meets the definition of formal training. It is unlikely that the provision of in-kind resources such as data or samples would be sufficient enough to allow for experiential learning related to a student's HDR as DESE would generally expect this would entail some form of support and/or active engagement from the research end-user.
Q: I completed formal training on end-user engagement at ANU or another University or Institution, is this something I should include and if so what industry do I select?
A: Yes, formal training can be provided by the ANU or any other HEP or research end user organisation, you should record this as an activity if the training focusses on preparing you for work with a research end-user. The industry you should select is 'other' and in the provide further details section you can enter the name of the organisation and industry category such as 'ANU', 'HEP'.
Data, field work, and other in kind-resources?
Q: If I have received in kind-resources such as data or samples from a research end-user (or their site) does this count as a research end-user engagement activity and if so which category? What about field-work?
A: Yes in kind-resources such as data, samples or field-work would fall under the category of 'Other commercialisation and engagement activities' so long as it has enabled 'experiential learning' related to your PhD research. Note the research end-user organisation also needs to directly use or directly benefit from the output, outcome or results of the research you have conducted).
Note 'experiential learning' can be thought of as the process of learning through experience or active learning as opposed to passive learning. It may include items such as observations, interactions, decision making, analytical skills, problem solving skills, and reflection. It also is where Candidates can ' ... apply knowledge and conceptual understanding to real-world problems ... ' (Wurdinger & Carlson, 2010). This would fit well with practical research such as internships, field trips/work, and field research. Note the arrangement does not need to include a written agreement.
Publications or books?
Q: What about publications or books that may benefit certain communities or research end user organisations - would this fall into any of the research end-user engagement activities?
A: Data collected relates to engagement between HDR Candidates and research end-users, rather than specific research outputs. If an HDR student is collaborating with a research end-user and the result is a research publication or book, then yes, that collaboration could be eligible as one of the indicators as long as this engagement meets the definition for either internship, joint supervision, joint funding, formal training or other engagement activities. However, if an HDR student produces a publication or book without engagement from a research end-user which then results in subsequent benefits to certain communities or research end-user organisations then this would not be included in this data collection.
Q: Is CSIRO or Data 61 considered to be an affiliate of ANU?
A: No. As the CSIRO is a publically funded research agency (PFRA), it would not be considered as an affiliate/controlled entity/subsidiary organisation, and could therefore be a research end-user if it was directly benefiting from an HDR student's research. Data 61 is a sub-entity of the CSIRO, so similar arrangements would apply.
Q: I receive funding from Data61 (either a stipend, top up, or funding towards other research materials and activities). Am I jointly funded by a research end-user?
A: Yes. Data61/CSIRO is a publicly funded research agency and are classified as research end-users.
Q: I need further help or advice who can I contact?
A: After reading the information on this page, if you require further advice or technical support please contact GRO via an email to firstname.lastname@example.org or by a phone call to +61 2 6125 5777. Note: you may also seek advice from your College or School HDR administrators or HDR convenors/supervisory panel members around your engagement activities.