Muneeb Mohammed Alzghool1, Abdulrahman Mohammed Al-Bakiri2*
1Community and Mental Health Nursing Department, College of Nursing, King Saud University, Saudi Arabia.
2*Specialist Nurse, King Salman Hospital, Riyadh. Saudi Arabia.
Corresponding Author Details: Abdulrahman Mohammed Al-Bakiri, Specialist Nurse, King Salman Hospital, Riyadh. Saudi Arabia. E-mail: email@example.com
Received date: 20th August, 2018
Accepted date: 10th January, 2019
Published date: 21st January, 2019
Citation: Alzghool, M.M., & Al-Bakiri, A. M. (2019) The Future of Mental Health Nursing Practice In Saudi Arabia: A Delphi Study. J Comp Nurs Res Care 4(1):132.
Copyright: ©2019, This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Background: Mental health nursing education, research, and practice is well examined in North America and Europe. But, evidence in literature relating to the future of nursing practice in Saudi Arabia is very limited.
Purpose: The aim of this study was to explore the future directions of mental health nursing practice in Saudi Arabia by obtaining expert consensus and to recommend a future agenda for policymakers and researchers in the field of mental health nursing.
Methodology: A Delphi method was used. The data was collected over three rounds. For the first round, the researcher utilized openended question questionnaire. Following the content analysis of the open-ended questions (639) responses were elicited. A 33 items questionnaire was then constructed with five major categories and were utilized later during the second and third round of the study. The initial sample consisted of (N= 87 males) and (N=63 females), Saudi and Non-Saudi. The rank of panel members were staff nurses, nurse managers, lecturers, assistant professors, associate professors, and professors. The second roundconsisted of (N=140), while the third round consisted of(N=120) of the experts panel who agreed to continue in participating in the study.
Results: The results of the study showed that consensus among the experts was reached on 19 elements with agreement level of 80% or more. In conclusion, it was apparent that expert panel believes that there are many elements that should shape up the future of mental health nursing practice by 2025. However, less emphasis was apparent from the panel ratings on items such as the stigma associated with mental illness and mentally ill people.
Conclusions: The results of the study showed that 19 items have achieved 80% and over of consensus and agreement by the experts. Most of the remaining items reached an agreement level by the experts panel ranging between 60% to 79%, which suggests that the future directions of mental health nursing practice in Saudi Arabia is not only limited to those items that achieved consensus by 80% or more, and that policymakers and nurse researchers should consider both thoroughly.
Keywords: Mental Health Nursing – Nursing Practice, Saudi Arabia- A Delphi Study
By the late of the nineteenth century nursing as a profession began to emerge and evolved into a specialty with unique roles and functions in the twentieth century . This included not only the undergraduate studies but also extended to the postgraduate studies which ultimately provided knowledge that enabled nurses practice nursing profession with more confidence in providing quality care for their patients. Earlier many researchers pointed out that a successful implementation of aneffective health care system to provide adequate and high quality services for all; including a balance between preventive and curative services and improving managerial and administrative skills in health care facilities and training of Saudi health manpower should be considered and planned for appropriately .
Such remarks among other factors have motivated the government authorities to give more attention to mental health, mental health nursing, and mentally ill people with an emphasis on mental health care providers by developing a strategic plan that aims at improving mental health and services in Saudi Arabia . Thus, the Saudi government has passed a Mental Health Act (MHA) that focuses on a number of elements. Among these elements is the improvementof the level of access to mental health care services, ensuring the least restrictive level of care, and preserving the rights of mentally ill people to equal care . However, there are still a number of barriers that hinders mental health care services and treatment such as religious healing malpractices, gender segregation, and social and legal aspects .
While nurses in Saudi Arabia is still struggling to practice their profession based on qualification in psychiatric\mental health nursing, a new trends of care for mentally ill people emerged during the mid of the twentieth century. This was the movement toward treatingmentally ill in a community based settings which gained momentum in 1963 with the passing of the Community Mental Health Centers Construction Act. According to . Community mental health centers serve smaller geographic areas. She maintained thatthese centers provided emergency care, inpatient care, outpatient services, partial hospitalization, screening services, and education. Therefore, this policy which also known as deinstitutionalization reduced admissions to hospitals, and established community-based services as an alternative to hospital based care.Prior to introducing the deinstitutionalization care, the concept of psychiatry has changed dramatically. This imposed changes on mental health nursing specialty, namely expressing for a professional and real identity for mental health nursing and competent nurses with special skills in dealing with mentally ill people .
Although the role of nurses in many countries particularly the developing countries remained custodial and almost similar to those tasks of one century ago, which is keeping mentally ill people safe and providing them with the basic human needs, many researchers believes that mental health nursing implies something more meaningful and can offer the clients more than a custodial role . This conception is in congruent with the definition offered by the American Psychiatric Nurses Association (APNA) which conceptualized mental health nursing asan area of nursing practice that is specialized and committed to promoting mental health through a well-defined nursing process .
The future demands of education and training were emphasized by . Who compared between England and Wales. They suggest that essential skills and knowledge in psychiatric mental health to protect and potentially enhance the therapeutic relationship is needed and existed with less attention given to Mental Health Act legislations. Willis commission also gave a careful attention to the current nursing workforce and the future of nursing workforce. The report concluded that most of the nurses who are delivering care for the next 10 years and more are already at work. Therefore, more attention must be given to continuing professional development . This future vision is based on the hope that the values, attitudes, and beliefs of psychiatric mental health nurses will lead to mental health nursing practice that promotes people`s health and well-being, and maintain a high quality of life through proper education and demonstration of skills .
In the United States, more than 80,000 registered nurses are working in mental health institutions, half of them are employed in mental health hospitals and the other half are community-based nurses. There are also more than 20,000 advanced practice psychiatric mental health nurses with graduate degrees. In Saudi Arabia, the first psychiatric hospital was established in the City of Taif in 1952. Since that, there was slow but steady progress in the develop ment of mental health services in Saudi Arabia. Subsequently , the Saudi Ministry of Health become the main provider of public mental health services which follows the core themes of the (WHO) in developing mental health services .
Although their role was in most is to carry out the psychiatrists orders, the history of psychiatric\mental health nursing practice in Saudi Arabia can belinked to the beginning of psychiatric services which started early in 1950s . Later on, a (Central Nursing Committee) was established. This was in 1987 at the (MOH) with the aim of improving the quality of nursing care and to recruit more Saudis to the nursing profession. Prior to 1987, there was no representation or formal voice for nurses at the national level hence the profession was dominated by physicians .
This study to the knowledge of the researcher represents a first attempt towards exploring the provision and future directions of mental health nursing practice in Saudi Arabia from an expert's perspective. The researcher, therefore, believes that this study is mainly derived its importance from its focus. In this case, the focus was the future.
In reviewing literature, it was apparent that nursing services are primarily provided by the public sector institutions, namely by ministry of health and hospitals under its authority. This motivated the researcher to try and shed some light on the practice domain of mental health nursing guided by the believe that this study will be of significant value not only for the ministry of health as an institution or the policymakers within it but also for all stake holders including nurses and other health team members who are working in public and privatehospitals.
The study aimed to explore the future directions of mental health nursing practice in Saudi Arabia by the year 2025. The following open-ended question was the essence of this study:
1 - What are the future directions of mental health nursing practice in Saudi Arabia by the year 2025?
This study aimed to explore future directions of mental health nursing practice in Saudi Arabia by the year 2025. Specifically, the study is aiming at:
1. Utilizing Delphi methodology to identify the nature and areas of future directions in the field of mental health nursing.
2. Obtaining expert consensus on the top ten areas that should shape up the future of mental health nursing practice.
3. Reporting a future agenda for researchers and policymakers.
Delphi technique was seen by the researchers as the most appropriate approach for collecting data concerning the aims of this study. This approach is believed to shed light on the aim of the study and be more helpful for educators, researchers, and practitioners in further understanding the expert perspectives.
Delphi method has been used widely in business, communications, education, and nursing research . According to , The Delphi technique has been utilized by many nurse researchers in a wide variety of studies.  used a Delphi of three rounds to gain consensus on how multi-disciplinary clinical staff view the development of community mental health centers over the course of 2 years in North Staffordshire.  a used a three rounded Delphi study to identify clinical nursing research priorities. The approach found to be useful and can be used in nursing policy and administration . And in medical education research .
Delphi technique involves multiple rounds to reach consensus without the need for the face-to-face meeting, therefore; it is a time and cost saving [21,22]. The use of a panel of 'experts' for obtaining data, the use of questionnaires and interviews and research problems does not lend itself to precise analytical techniques but can benefit from subjective responses on a collective basis [23,24]. Quantitative or qualitative scales may be used, and the Questionnaires may or may not include open-ended questions .
First round: Delphi process traditionally begins with an openended questionnaire. The open-ended questionnaire serves as the cornerstone of eliciting specific information about a content area from the subjects. After receiving subjects’ responses, investigator's need to convert the collected information into a well-structured questionnaire. The questionnaire then is used as the survey instrument for the second round [26,27].
Second Round: Each participant receives the second questionnaire and is asked to review the items summarized by the researchers based on the information provided in the first round. Accordingly, panelists may be required to rate or "rank-order" .
Third Round:  points out that Delphi panelist receives a questionnaire that includes the items and ratings summarized by the researchers in the second round and are asked to rerate his judgments or to determine the reasons for remaining outside the consensus.
Sample: Delphi study focuses more on the degree of expert consensus according to their knowledge and experience to be representative of any population, therefore, an expert who has a deep understanding of the issues is the most appropriate requirements in the process of selection of expertise panel [30,31].
The selection of the sample involves nonprobability sampling techniques, either purposive sampling or criterion sampling. Expert panel are not selected randomly, and therefore; generalization cannot be made. However, they are selected for a purpose, to apply their knowledge to a certain problem on the basis of criteria, which are developed from the nature of the problem under investigation [32,33].
The researchers approached and invited 200 expertise. The sample size who agreed to participate and included in the first round was (150) female and male, Saudi and others, staff nurses, nurse managers, lecturers, assistant professors, associate professors, and professors.
The whole study was carried out over one year. It started in January 2016 and completed by the end of the same year. Following the analysis of the open ended question, the first round conducted between (12-2-2016 and 26-2-2016) over two week time, and followed by modifications on the thematic responses, expertise assessment, and re-modification based on expertise comments. The second round then started andcarried out also over two weeks(15-5-2016 and 1-6-2016). Following the analysis of round two, round three questionnaire then distributed between(1-10-2016 and 15-10-2016). It also took (2 weeks) time.
Approval of (Institutional Review Board, Approval Number Federal Wide Assurance NIH, USA: FWA00018774) conduction was obtained from King Fahad Medical City (Ministry of Health). A permission of the nursing college at King Saud University was also granted to facilitate approaching the panel members and carrying out the study.
The questionnaires were distributed to two of groups of expertise. The first consist of faculty members from eight Saudi universities, and the second consist of nursing staff from three major Saudi mental health hospitals. The questionnaires were sent by email to the faculties and directly by the
For the purpose of this round, the researchers designed a questionnaire. This questionnaire contains, demographic details about the participants and the one open-ended question. This was
1- Based on your expert knowledge, what are the most important changes that you believe should shape up the future of mental health nursing practice during the next ten years in Saudi Arabia?
In this round, the expertise panel (n = 150) reviewed one openended questionwhich asked participants to list their perspectives on mental health nursing practice issues that should shape up the future of mental health nursing in Saudi Arabia by the year 2025, and were asked to provide at least five responses. The time frame to complete the questionnaire for this round was 2 weeks.
The five comments of participants were folded into(33) statements\ themes on practice during the analysis of round one. The initial number of total responses on practice were (960) responses. Most of these responses were clear and relevant to the open-ended question. However, some of the responses found to be irrelevant and unclear (38 statement), therefore, the researchers revised all responses and omitted (321) responses.Table Two illustrates the frequency of responses:
Five main categories emerged from the above themes. These categories and subthemes will be reflected in the construction of the questionnaire which will be used in rounds two of the study.
The researchers constructed the questionnaire which will be used in the second round of the study based on participants' perspectives elicited from the first round. The questionnaire contains the elements that should shape up the future of mental health nursing practice. The following domains was derived:
1 - Clinical skills and Work Environment (14) items 42%
2 - Administrative and Leadership Style (6) items 18 %
3 - Support and Rewarding Systems (4) items 12 %
4 - Culture and Societal (4) items 12 %
5 - Competency and Communication Skills (5) 15 %
In this study, the content validity of the second round questionnaire was conducted by sending the questionnaire to (10) expertise panel working in nursing colleges to see their suggestions, to evaluate, to and identify whether the questions agreed with the scope of the items and the extent to which these items reflect clearly the concept of the research problem.
In Round two, the researcherssend and distributed (150) questionnaires to the participants who responded to round one openend questionnaire. Out of the (150) only (140) returned their rating to the second round questionnaire, and (10) of them declined without specific reason.
Following the completion of round two analyses, the questionnaire was returned to the same expertise who participated in round two with their ratings. The expertise during this round (Round three) was also asked to add optional comments on each statement and to add any ideas for further consideration if not have been addressed in round two.
In Round three the researchers distributed (140) questionnaire to the who responded on the questionnaire in the round two. In this round only (120) participants returned, and (20) of them were declined.
The findings of the round three ispresented as follow; The Mean Scores, Standard Deviations and Level Agreement for each individual expert’s response on the category and themes of each statement. Table: 4 illustrates:
The researchers in this study has chosen to terminate the current study after Round Three because 29 statements contained more than 80% agreement indicating a high level of consensus regarding the importance of statements rated in the study.
The expertise agreement and consensus on category one were statistically significant. The mean score for Round 2 was (3.93), and the mean for Round 3 was (3.90). Seven items (4, 6, 7, 8, 9, 13 and 14) have below 80%. This indicates that they remained areas of disagreement among experts in the study concerning. The statements that have had the level of agreement over 80% were (1, 2, 3, 5, 10, 11 and 12) rated as ranging from ''agree'' to "strong agreement " by experts.
Expertise perspective on all items would influence positively the future of mental health nursing especially regarding the administrative and leadership style. The expertise reached consensus on category two which was statistically significant. The overall mean scores of this category for Round 2 was (4.35), and the overall mean score for Round 3 was (4.03). All items level agreement up to 80% except item (15). Items (16, 17, 18, 19, and 20) and rated as ranging ‘‘Agree’’ to “strong agree” by experts.
The experts' agreement and consensus on category three were statistically significant. The overall mean score of this category for round 2 was (4.21), and for Round 3 (3.79). Three items (22, 23, and 24) have the agreement under 80%. This indicates that there remain areas of disagreement among experts in the study concerning the importance of these factors in relation to influence on the future of mental health nursing in Saudi Arabia especially in part of the support and rewarding systems. Only one item (21) has agreement level above 80% and rated “strong agree” by experts.
Experts' agreement and consensus on this category were statistically significant. The overall mean score of this category in round 2 was (4.45), and the overall mean score in round 3 was (4.83). All items have 100% agreement and rated as “strongly agree” by experts.
Expertise agreement and consensus on category five were statistically significant. The overall mean score of this category in round 2 was (3.53), and the overall mean score for Round 3 was (3.92). Three items (31, 32, and 33) have had agreement level under 80%. This indicates that there remain areas of disagreement among experts in the study concerning the factors influencing the future of mental health nursing in Saudi Arabia especially in part of competency and communication skills. The statements are (29 and 30) have 100% agreement and rated as "strongly agree" by experts.
The results showed that the panel members agreed and reached consensus with 80% or more. The researcher will discuss highest rated items (Above 80% Agreement) in each Category of the practice domains respectively:
In round one, items" Nurses should achieve higher levels of education and training to practice mental health nursing" and "Clinical training program and workshop should be attended by mental health nurses"were highly rated by the expert panel. The mean scores of these items in round two were (4.93, 4.86) with a standard deviation (0.26, 0.35), and in the third round the mean scores were (4.83, 5) and with a standard deviation is (0.37, 0). These results are in agreement with the results from previous research studies conducted by . This included not only the undergraduate studies, but also extended to the postgraduate studies which ultimately provided nurses with the necessary knowledge and skills that enabled nurses to practice nursing profession with more confidence in providing quality care for their patients.  stated in their study on the need for training in-service for nurses in mental health in Saudi Arabia. Therefore, argued by the researcher in this study that is important to give more attention to clinical training in mental health nursing to enable nurses in Saudi Arabia providing quality care.
The results in this domain also revealed that other factors would influence the future of mental health nursing practice. The expertise panel gave high rating to items ''Evidence-based practice to deliver quality care, Encourage simulation training in mental health nursing, Reductions in the work hours, Overcome barriers and Recreational activities to improve patient safety'' suggesting that these elements should be influential in the domain of mental health nursing practice during the next ten years in Saudi Arabia.
The items "Clear job description and scope of mental health nursing practice'' were highly rated in round two and three. Experts in this study believe also that collaboration of mental health nurses with other health team members, more effective role for the nurses in charge to develop the skills of employees, justice, enforce reward and Punishment regulation, are all important issues that should shape the future of mental health nursing in Saudi Arabia. The researcher in this study argues that for such elements to be implemented effectively research studies must be carried out on how best such elements may be helpful to nursing and nurses.
The item "More nurse practitioners should work in mental health setting" was highly ranked by expertise panel. The mean scores for this statement were (4.89, 5). The results concerning this item showing an agreement with studies conducted by  which indicated that the increase in number of qualified of nursing staff in hospitals are important issues that will shape up the future of mental health nursing in Saudi Arabia.
The analysis of the three rounds in this study indicated that ''Less social stigmatization towards mentally ill people should be encouraged'' was rated to the highest level of agreement among the expert panel.The item "Change the image of nursing in Saudi society" was rated highly in round two and three with mean scores of (4.29-5). The expertise in this study believes that the Participation of nurses in mental health awareness campaigns and Introduce a specific patient bill of rights are the important element that should shape the future of mental health nursing in Saudi Arabia. Although it is well known as reviewed earlier in chapter two that the government gave attention to health care system with a huge amount of money expenditure, the researcher in this study argues regulations such as the bill of patients' rights in mental health needs to be reinforced and applied all over the country.
Competent nurses are those nurses who are able to equip themselves with the highest level of knowledge and clinical skills. It is argued by the researcher in this study that mental health nurses will not be able to build up a therapeutic relationship with their patients without improving their communication skills.
The results of the three rounds in this study revealed that the statement "the necessity of building a relationship between the patient and nurse" and "follow up developments in mental health nursing globally" were among the highest rated items by the expert panel. Overall, this rating is congruent with the findings of [5,13]. The findings of this study are based on the perspectives of nurses who are working in the field mental health nursing with a variety of qualifications and add another debate about the future directions of mental health nursing practice.
The results of the study showed that 19 items have achieved 80% and over of consensus and agreement by the experts. Most of the remaining items reached an agreement level by the experts panel ranging between 60% to 79%, which suggests that the future directions of mental health nursing practice in Saudi Arabia is not only limited to those items that achieved consensus by 80% or more, and that policymakers and nurse researchers should consider both thoroughly.
The combined consensus among Saudi and Non-Saudi related to the future direction of mental health nursing Practice were:
1. Nurses should achieve higher levels of education and training to practice mental health nursing. 2. Clinical training program and workshop should be attended by mental health nurses. 3. Evidence-based practice to deliver quality care. 4. Encourage simulation training in mental health nursing. 5. Reductions in the work hours of nurses to improve patient safety. 6. Overcome barriers to proper standardized practice by mental health nursing. 7. Recreational activities for mental health nurses. 8. Clear job description and scope of mental health nursing practice. 9. Expand opportunities for mental health nurses to lead collaboration with other health team members. 10. The more effective role for the nurses in charge to develop the skills of employees. 11. Enforce Reward and Punishment regulation. 12. Justice and equality among nursing staff. 13. More nurse practitioners should work in the mental health setting. 14. Less social stigmatization towards mentally ill people should be encouraged. 15. Participation of nurse in mental health awareness campaigns. 16. Introduce specific patient bill of rights. 17. More positive attitude towards mental health nursing should be encouraged. 18. Follow up developments in mental health nursing globally. 19. Encourage a more therapeutic relationship between mentally ill people and mental health nurses.
The results of the study may contribute to the formalization of the new vision of the philosophies and strategies to improve performance of mental health nursesin Saudi Arabia by increasing numbers of nurses, pay attention to the training of nurses during work, acquaint them with all new about mental health globally, increase bonuses and incentives in order to students to enroll in nursing especially mental health nursing. It is hoped that the study will help in the development of mental health nursing and interdisciplinary research to improve the nursing practice conditions.
There were several limitations to this study. The main one was related to the selection of participants on the basis of experience in working in mental health setting and specialty. Many of the expertise was not prepared as mental health nurses but rather as a general nurse with experience in working in mental health settings. Many of them were not also originally from Saudi Arabia and not fully oriented about the culture of Saudi Arabia which may constitute another limitation.
The researchers extend their Acknowledgment to King Saud University for their academic and moral support, and for the Saudi ministry of health for their financial support.
Stuart GW (2014). Principles and practice of psychiatric nursing, 10th edition Elsevier Health Sciences. View
Sebai Z A, Milaat WA, Al-Zulaibani AA (2001). Health care services in Saudi Arabia: past, present and future. J fam community med 8: 19.View
Al-Habeeb AA, Qureshi NA (2007). Mental and Social Health Atlas I in Saudi Arabia: 2007-08. E Mediterr Health J 16: 570- 577. View
World Health Organization, & International Council of Nurses. (2007). Atlas: nurses in mental health 2007. World Health Organization.View
Koenig HG, Al Zaben F, Sehlo MG, Khalifa DA, Al Ahwal M S et al. (2014). Mental Health Care in Saudi Arabia: Past, Present and Future. Open J Psychiat, 4: 113–130.View
Videbeck S (2013). Psychiatric-mental health nursing. 6th edition , Lippincott Williams & Wilkins.
Hemingway S, Sutton M, Heraty I, Broadbent E, Elliot R et al. (2011). Interviews for mental health nursing students: key considerations for the panel. Mental Health Nursing (Online), 31: 202.
Tratnack SA, O’Neill CM, Graham P (2011). Immersion Experience in Undergraduate Psychiatric Mental Health Nursing. J Nurs Edu, 50: 532–535. View
American Nurses Association. (2010). Nursing: Scope and standards of practice. Nursesbooks. Org.View
Hurley J, Linsley P (2006). Proposed changes to the Mental Health Act of England and Wales: research indicating future educational and training needs for mental health nurses. J psychiat mental health nurs, 13: 48-54.View
Willis P (2012). Quality with Compassion: the future of nursing education. Report of the Willis commission on Nursing Education. Royal College of Nursing on behalf of the independent Willis Commission on Nursing Education 56.
Callaghan P, Owen S (2005). Psychiatric and mental health nursing: past, present and future. J Psychiat and Mental Health Nurs, 12: 639–641.View
Qureshi NA, AlHabeeb, Koenig H (2013). Mental health system in Saudi Arabia: an overview. Neuropsychiatric Disease and Treatment.View
Aboul-Enein FH (2002). Personal contemporary observations of nursing care in Saudi Arabia. Int J Nurs Pract, 8: 228–230.View
Watson T (2008). Public relations research priorities: a Delphi study. J Comm Manage, 12: 104–123.View
Hasson F, Keeney S, McKenna H (2001). Research guidelines for the Delphi survey technique. J Adv Nurs, 32: 1008–1015.View
Beech B (1997). Studying the future: A Delphi survey of how multi-disciplinary clinical staff views the likely development of two community mental health centers over the course of the next two years. J Adv Nurs, 25: 331-338. View
Bond S, Bond J (1982). A Delphi survey of clinical nursing research priorities. J Adv Nurs, 7: 565–575. View
Whitman NI (1990). The Committee Meeting Alternative; Using the Delphi Technique. J Nurs Admin, 20: 30-36. View
Humphrey-Murto S, Varpio L, Wood TJ, Gonsalves C, Ufholz LA, (2017). The Use of the Delphi and Other Consensus Group Methods in Medical Education Research: A Review. Academic Medicine, 92: 1491-1498. View
Tsaroucha A, Boath E, Porteous E, Wright A (2015). Using a modified Delphi Method to develop a new advanced accreditation award (‘Triple A’) in money advice practice. Innovative Practice in Higher Education, 2(2). View
Jorm AF (2015). Using the Delphi expert consensus method in mental health research. Aus & NZ J Psychiat, 49(10), 887-897. View
JSeker SE (2015). Computerized argument Delphi technique. IEEE Access, 3: 368-380. View
Prak M, Wivatvanit S (2018). The development of neonatal nursing standards of practice for Cambodia: a Delphi study. J Health Res, 32: 12-23. View
Somerville JA (2008). Effective use of the Delphi process in research: Its characteristics, strengths and limitations. Corvallís (Oregon). View
Custer RL, Scarcella JA, Stewart BR (1999). The modified Delphi technique-A rotational modification. J Career and Technical Edu, 15. View
Kim CH, Yeo K (2018). Beyond Consensus: A Review Of Delphi Research Published In Malaysian Social Science Journals. Int J Business & Society, 19. View
Ludwig B (1997). Predicting the future: Have you considered using the Delphi methodology? Journal of Extension, 35 : 1-4. View
Hsu CC, Sandford BA (2007). The Delphi technique: making sense of consensus. Practical Assessment, Research & Evaluation, 12: 1–8. View
Hsu CC, Sandford BA (2007). The Delphi technique: making sense of consensus. Practical Assessment, Research & Evaluation, 12: 1–8. View
Lis R, Sakata V, Lien O (2017). How to choose? Using the Delphi method to develop consensus triggers and indicators for disaster response. Disaster medicine and public health preparedness, 11: 467-472. View
Page A, Potter K, Clifford R, McLachlan A, Etherton-Beer C. (2015). Prescribing for Australians living with dementia: study protocol using the Delphi technique. BMJ open, 5. View
Kangasniemi M, Arala K, Becker E, Suutarla A, Haapa T et al. (2017). The development of ethical guidelines for nurses’ collegiality using the Delphi method. Nursing ethics, 24: 538- 555. View
Tumulty G (2001). Professional development of nursing in Saudi Arabia. Journal of Nursing Scholarship, 33: 285–290. View