Psychometrician

Term: Temporary, Full-Time (ending March 31, 2026)
Work Location:  Hybrid, Vancouver B.C. (in-office 2 days per week)

Who we are

Inspire Global Assessments (Inspire), formally known as the Nursing Community Assessment Services (NCAS), is a branded service of the BC College of Nurses and Midwives. We’re working with nursing regulators and governments across the country to address the acute nursing shortage. Inspire is a unique assessment service that supports the transition of internationally educated health care professionals into the global work force.

The British Columbia College of Nurses & Midwives (“BCCNM”) is the college empowered under the Health Professions Act to regulate the practice of all licensed practical nurses, nurse practitioners, registered midwives, registered nurses, and registered psychiatric nurses in British Columbia. Regulation helps to protect the public by ensuring that professional care or service received by the public is competent, ethical, and meets the standards that society views as acceptable.

As western Canada’s largest health profession regulator, we believe a diverse and inclusive team enriches our efforts to protect the public. We welcome applications from all who reflect of the communities we serve, and especially encourage Indigenous Peoples and members of equity-seeking groups to apply. We believe diverse perspectives and experiences bring both innovation and better outcomes to the work we do and the decisions we make.

BCCNM’s commitment to Indigenous Specific Anti-Racism

BCCNM is committed to working with Indigenous Peoples and health system partners to support Indigenous-specific anti-racist practices in the health system. As the largest health regulator in Western Canada, BCCNM has the opportunity to influence the health professionals we regulate, as well as the broader health-care system, to break the cycle of racism, and improve health outcomes for Indigenous Peoples. BCCNM has pledged to become culturally safe and anti-racist, and to support the health professionals we regulate to do the same.

What we offer you​​

We promise to provide you with rewarding work that challenges you. You will be part of a passionate team contributing to our important mandate of protecting the public. As a member of the team, you can expect to be working in a collaborative, team-based environment, and treated in a respectful and professional manner.

The expected starting base salary for this position is $75,000 – $85,000 annually, depending on a variety of factors including qualifications and experience.

What you’ll be doing

Reporting to the Manager of Assessment, Innovation and Psychometrics (AIP), the Psychometrician supports Inspire Global Assessments with ongoing and future assessment activities, including those related to BCCNM’s broader examination assessment needs. The position focuses on providing psychometric expertise to the continuous refinement and evolution of Inspire’s multi-faceted triple-track assessment. The Psychometrician supports a range of activities consisting of: data management, code development, scoring methodologies, design of assessment content, and the analysis of items. They also perform quality assurance, statistical data analysis, and storage data measures.

Key Responsibilities

• Lead the development, refinement, and analysis of performance assessment instruments, conventional and innovative test methodologies, and standard setting and validation activities associated with competency-based assessments.
• Participate and collaborate in the scoring, development of blueprints, case development, modification, and field-testing analytics as well as related activities.
• Generate performance reports, perform quality checks, and provide results to key stakeholders (written or verbally).
• Interface with multiple stakeholders and technology providers, facilitating discussions as needed.
• Create and implement quality-control policies and procedures, and develop, implement, and evaluate standard-setting and product-improvement processes.
• Develop, manage, and monitor item exposure rates and statistical anomalies.
• Provide psychometric consultation and problem-solving in support of assessment operations, including using IRT, item analysis, automatized scoring, forms construction and equating, reliability, validity analyses, etc.
• Perform regular data management and processing tasks, including extracting, cleaning, organizing, and transforming data.
• Design and apply surveys to collect data to inform assessment effectiveness and efficiency.
• Maintain and/or provide oversight of testing platforms, vendors, and databases.
• Develop documentation to support assessment infrastructure along with public and technical reports on the assessments.
• Work to integrate AI and intelligent algorithms to analyze and predict future events on tasks, virtual performance-based assessments, and Computational Psychometrics.
• Carry out independent research and present research findings, representing the organization and sharing innovations at professional conferences and/or through publication of research articles.
• Collaborate with the Assessment Developer in the development and refinement of content as needed. Provide feedback, troubleshoot content materials and activities, and identify and mitigate risks to ensure integrity and validity of assessment content.
• Under the direction of the Manager, AIP, support the Assessment Developer in facilitating in-person and online item writing, pilot and field testing, and standard setting sessions as needed. 
• Perform other job-related duties and tasks as assigned.
• Demonstrates a commitment to ongoing learning related to Indigenous cultural safety and humility and supporting organizational actions towards addressing indigenous-specific racism in BC’s health care system.
• Fosters and maintains an organizational culture that promotes equity, diversity and inclusion, mutual respect, teamwork, and service excellence.

What you’ll bring

• A master’s degree, Ph.D. preferred, in Measurement and Evaluation, Cognitive, Quantitative Language, or Computer Science (with a concentration in Psychometrics), or the educational equivalent in a closely related quantitative field. A combination of related education and proven experience may be considered.
• At least five years of work experience in an applied measurement statistical analysis or computer science environment and at least three years in designing and conducting psychometric and statistical analysis in a test environment.
• Advanced knowledge of statistics and data modelling (e.g., sampling theory and methods, mixed effects models, exploratory data analysis, computational approaches to parameter estimation, multilevel modeling, Bayesian methods, and experimental and quasi-experimental methods for causal inference).
• Expertise with a variety of statistical models used in testing to manage and transform data, design and conduct research, simulations, and generate performance and technical reports (eg: classical test theory, item response theory, generalizability theory, factor analysis, HLM, norms development, and item response theory).
• Demonstrated ability to write and modify code and programs using advanced statistical analysis software (R, Python and FlexMIRT preferred; SAS, SPSS, Mplus, Winsteps, BILOG and/or PARSCALE) and to retrieve data from databases (SQL).
• Familiarity with professional testing standards (e.g., ISO 17024, NCCA, Joint Technical Standards, ITC Guidelines, INACSL, and the Standards for Educational & Psychological Testing) and ability to implement policies and procedures.
• Understanding of computational methods using AI and Natural Language Processing.
• Experience with testing programs with small to medium volumes and ability to transfer knowledge and expertise to a complex integrated assessment service.
• Ability to identify, understand, and value relevant social characteristics of the assessment context and clients to support developing and applying bias, sensitivity, and accessibility policies and procedures to enhance the validity and effectiveness of the assessment.
• Strong interpersonal and communication skills. Experience working with external vendors and providing group facilitation is an asset.
• Demonstrated project management skills with ability to prepare analytic plans, monitor progress, and track timelines.
• Exceptional research skills and a curiosity for staying current with the field of assessment as well as ability to prepare effective visual presentations of complex data.
• Demonstrated initiative and problem-solving skills with strong organizational skills and the ability to meet deadlines in a fast-paced environment.
• Ability to work independently as well as collaboratively within a team.
• Fluency in French is an asset.

How to Apply

If you are interested in this position, please apply online by October 20, 2024. This opportunity will remain posted until filled; however, priority consideration will be given to those who apply by the deadline. To see a full list of our current opportunities or to learn more about working at the BC College of Nurses & Midwives, please visit our website at Careers at BCCNM.

Collection Notice

To apply to a job posting, you need to create an online account with BCCNM. To create an online account as well apply for a job posting, you’ll be asked to provide personal information.

The BC College of Nurses and Midwives (BCCNM) collects your personal information for the purposes of creating your online account as well as for recruiting, evaluating, and selecting employees. The legal authority for collecting this information is under section 26 British Columbia’s Freedom of Information and Protection of Privacy Act.   If you have any questions, please contact [email protected].

Please note: BCCNM no longer requires staff, contractors, board and committee members, and volunteers to provide an attestation of vaccination related to the Covid-19 pandemic.

In the event of a Public Health Order being issued, the College is committed to full compliance with all directives and guidelines set forth by the Provincial Health Officer (PHO) to ensure the safety and well-being of our employees and the community partners we work with, consistent with the duty of the College to serve and protect the public.