Evidence Synthesis

First, it is helpful to define some terms. According to PubMed's MeSH (Medical Subject Headings):

Here is a graphic from IBM that helps contextualize some of these terms in relation to each other and how they evolved over time.

"AI" is often used to refer to generative AI and large language models such as ChatGPT. However, there are other types of automation and artificial intelligence outside of generative AI. Examples include relevance ranking in search engines, spell check, and content recommendations on platforms such as YouTube and TikTok. This page contains information about both generative AI and other types of artificial intelligence and automation. 

While generative AI can be a useful tool, it's important to use it responsibly. The sources below provide some information on generative AI ethics and considerations for use. A few areas to be aware of are: bias, privacy, environmental impact, and critical thinking. 

Some tools to automate or semi-automate parts of the evidence synthesis process can be used with caution. There are no set, universal rules for using automation tools, though some journals do provide their own guidelines on AI use, and you should always report any tools you used and how you used them in the methods section of your manuscript. 

A major consideration in using AI, generative AI, or automation tools for evidence synthesis is reproducibility. AI-powered research tools may advertise the ability to type in a natural-language statement or question to receive search results rather than having to construct a Boolean search (the one with all the ANDs and ORs), but if the same question does not yield the exact same results every time, it is not a reproducible search. It also raises the question of whether searches like this retrieve all the available evidence, which is the goal of evidence synthesis. Similar problems can occur with other steps, such as using AI or automation tools for relevance ranking in screening (assuming you stop screening while there are still articles left) and data extraction. 

Here are some resources and articles on automation in evidence synthesis.