Starting point
If you have worked through the personal course, you already know what an AI chatbot is and where it gets things wrong. This page picks up from there and asks a narrower question: what do these tools do for you as a researcher, and where is the line between useful and risky?
The honest summary is that AI can take real work off your plate, drafting, summarising, restructuring, explaining, and it can do it quickly. It can also produce confident, professional-looking output that is wrong, and in research a wrong answer that looks right is worse than no answer at all. The skill this course builds is using the speed without inheriting the errors.
AI is a fast assistant, not a reliable authority. The faster it works, the more your judgement matters, not less.
What these tools are, at research depth
The tools you will meet are the same general-purpose chatbots (ChatGPT, Claude, Gemini, Microsoft Copilot), plus a layer of specialist research tools built on top of them (Elicit, Consensus, Perplexity, NotebookLM and others, covered on the literature page). They all rest on the same engine: a model that predicts likely text from an enormous amount of training data. That single fact explains both what they are good at and where they fail.
Two practical points are worth carrying into every research use:
They generate, they do not look up. Unless a tool is explicitly searching a database or your own uploaded documents, it is writing from patterns, not retrieving facts. This is why it can invent a reference that looks perfect.
They are built to be agreeable. Models tend to go along with a premise you put in your prompt rather than challenge it. Research described this "sycophancy" affecting a majority of interactions across major models (industry testing, 2025), so a leading question often gets you the answer you hinted at, not the accurate one.
Where AI fits the research workflow
It helps to think of AI as useful at specific points in your work, not as a single tool you point at everything.
- Understanding. Turning a dense paper, a policy, or a statistical output into plain language; explaining a method you have not used before.
- Drafting. First drafts of plain-language summaries, reports, letters, grant sections, and presentations, which you then correct and shape.
- Restructuring. Reworking your own writing for a different audience, length, or reading level.
- Literature and evidence. Helping discover, screen and extract from papers, with strong limits (see the literature page).
- Data work. Some tools run real code on data you upload, for cleaning, exploring and charting, which is genuine computation rather than guessing.
- Admin. Meeting notes, agendas, plain-language consent material drafts, and the everyday writing that eats time.
What does not belong on this list is anything where the tool is the final authority: deciding what is true, what is culturally appropriate, or what a community wants. Those stay with people.
Where it earns its keep
Used well, AI gives the most back on tasks where you can check the output yourself.
Plain-language translation. Moving between technical and everyday language, in both directions, faster than by hand. This is genuinely useful for community-facing research material, as long as a person reviews the result.
First drafts. A blank page is slow; a draft to react to is fast. Researchers report substantial time savings on literature-review and grant writing sections (Clark et al., 2025), with the work shifting from writing to checking.
Summarising what you give it. When you paste in the actual text, a meeting transcript, a report, a set of abstracts, and ask for a summary, the tool is working from your material, which is far safer than asking it to recall facts from memory.
Structured extraction. Pulling information into a consistent table from text you provide, which is the basis of the literature-extraction tools.
Where it fails, at research stakes
This is the part that matters most in a research role, because the failures are quiet and confident.
It invents references. A 2025 study in JMIR Mental Health found that with one widely used model, nearly two-thirds of generated citations were fabricated or contained errors, and that the rate was worse for niche topics than well-known ones. Every reference an AI gives you must be checked against a real database before it goes anywhere near your work. The verification page covers how.
It makes things up even in careful settings. Hallucination is a built-in property of these models, not a bug that will be patched away. A 2025 study in npj Digital Medicine, across nearly 13,000 clinician-checked sentences, still found a small but real rate of invented and omitted detail in clinical documentation. Small and real both matter: AI output is a starting point, never a finished product.
It cannot be reproduced exactly. The same prompt can give different answers on different days, and models change version often. If you use AI in a piece of research, record the tool, the version, the date, and the exact prompt, so your method is transparent.
It is weak where the published record is thin. On Aboriginal and Torres Strait Islander people, places, language and community knowledge, much of what matters was never written into the text these models learned from, or was written by outsiders. The tool will still answer confidently. Treat local and cultural detail as something to check with the people who hold it, not something to take from an AI.
The rule from the personal course holds, with higher stakes here: helpful is not the same as truthful. A polished answer is not evidence that it is correct.
A note on AI scribes
One tool you may meet in a health setting is the AI scribe: software that listens to a consultation through "ambient" recording and turns the conversation into a draft clinical note, summary or letter. Used well, scribes can cut documentation time and let a practitioner pay more attention to the person in front of them.
They also raise the same issues as every other AI tool, sharpened by the clinical setting. The practitioner stays accountable for the note and must review it for accuracy; patient consent should be obtained and recorded; and only scribes built and approved for clinical use in Australia should be used, within your organisation's privacy and cybersecurity rules. Because a scribe handles identifiable health information, the data-handling rules on the responsible-data page apply directly. If a scribe is on the table where you work, that is a decision for the health service and its governance, not something to switch on individually.
A first word on which tool, and your data
Not all accounts are equal. Free public tools generally offer the least protection for what you type; some paid tiers commit to not training on your inputs; and workplace or enterprise accounts set up by your organisation are built to keep data inside. A tool that grounds its answers only in documents you upload (such as NotebookLM) also reduces invented content. The full treatment, what you can and cannot put into a tool, de-identification, and the Privacy Act and NT Information Act, is on the responsible-data page. The short version for now: match the tool to the sensitivity of the information, and never put community, cultural or identifying data into a public tool.
Questions of Indigenous data sovereignty, ICIP and research ethics have their own page; they sit at the centre of this course, not at the edge.
Glossary
- LLM (Large Language Model)
- the kind of AI behind a chatbot; a program trained on a lot of text to predict language.
- Generative tool
- software that produces new text, images or audio, rather than retrieving stored facts.
- Hallucination
- when an AI states something false as if it were true; a built-in property of these models, not a fixable bug.
- Citation fabrication
- when an AI invents a reference, or gets the details of a real one wrong.
- Sycophancy
- the tendency of a model to agree with a premise in your prompt rather than correct it.
- Reproducibility
- being able to get the same result again; limited with AI because outputs vary and models change.
- AI scribe
- software that records a clinical consultation and drafts a note from it; the clinician stays responsible for the note.
- NotebookLM
- a tool that answers only from documents you upload, which reduces invented content.
- Enterprise / workplace account
- an AI account set up by an organisation so that data stays within its control; safer for work than a free personal account.
Take it with you: a one-page cheat sheet of this material is available to download from the button at the top of this page.