Generative artificial intelligence refers to a type of AI tool that can produce content in response to user prompts. These rapidly evolving technologies are transforming operations across both the public and private sectors, and while organizations are moving quickly to adopt genAI, there are still concerns that must be considered as these tools are deployed.
Today, AI alone “is never going to get you 100 percent of the way through a task,” according to Procurement Sciences Inc. CEO and Founder Christian Ferreira.
In its current state, he said, genAI still requires a human-in-the-loop for “correcting it and adjusting its path.”
“If you are the architect and the AI is your builder, it’s going to build the building based on the blueprints you write, so the instructions you give it. But if you leave details out such as the tones, the lengths, things like that, the AI is going to go in there and use its own experience,” Ferreira said during GovCon Wire’s Generative AI: The Game Changer You Can’t Afford to Ignore Webinar on Wednesday.
This type of relationship is necessary for preventing some of AI’s potential shortcomings, one of which is issues with data privacy. GenAI models take in vast amounts of data for training and are widely available to different users, so individuals should be “very cautious around data privacy, especially with any sort of work-related documents or personal-related documents,” he said.
Another concern with genAI is accuracy. These tools, Ferreira noted, have a tendency to “hallucinate,” meaning that they can “make things up.”
“Some are better than others, but they’re not always accurate and it’s hard to spot hallucinations without having some sort of training behind the scenes,” he said.
Responses from GenAI are inherently influenced by the data they consume, which presents an additional challenge: bias.
“As you can imagine, the data that they’re trained on, these papers that people are writing, have cognitive biases already built into them that gets ingested by the AI,” said Ferreira.
This training method, he continued, “automatically” results in bias.
Though many companies have put regulations for AI use in place, Ferreira said it is still important for users to make sure that publicly available models are safe to use by checking with their organization’s cyber and information technology experts.
For more on AI, join GovCon Wire’s sister organization, the Potomac Officers Club, at its 2024 Navy Summit on August 15. This event will convene Navy officials and industry leaders to share their insights on the service branch’s top challenges and priorities and feature an insightful panel discussion about the role of AI in accomplishing these goals. To learn more and register to attend the summit, click here.
