Y’all Know That I am NOT an “ALL-IN”, no matter what, fan or supporter of “A.I.” I am a “Limited” Usages of “A.I.” as just 1/ONE Technology Tool. (It’s Not and “End All/Be All”).
AI Detectors Discriminate Against Non-Native Speakers, Says Stanford Research
AI Detectors Discriminate Against Non-Native Speakers, Says Stanford Research
By Erik Ofgang
published about 4 hours ago
The AI detectors got it wrong more than half the time when evaluating writing from non-Native English-speaking students, recent research found. For native English speakers, the findings were still far from perfect.
AI detection tools are supposed to help teachers identify AI-generated text but they have many problems, according to new research from Stanford. (Image credit: Getty Images)
A group of Stanford researchers recently decided to put AI detectors to the test, and if it was a graded assignment, the detection tools would have received an F.
“Our main finding is that current AI detectors are not reliable in that they can be easily fooled by changing prompts,” says James Zou, a Stanford professor and co-author of the paper based on the research. More significantly, he adds, “They have a tendency to mistakenly flag text written by non-native English speakers as AI-generated.”
This is bad news for those educators who have embraced AI detection sites as a necessary evil in the AI era of teaching. Here’s everything you need to know about how this research into bias in AI detectors was conducted and its implications for teachers.
How was this AI detection research conducted?
Zou and his co-authors were aware of the interest in third-party tools to detect whether text was written by ChatGPT or another AI tool, and wanted to scientifically evaluate any tool’s efficacy. To do that, the researchers evaluated seven unidentified but “widely used” AI detectors on 91 TOEFL (Test of English as a Foreign Language) essays from a Chinese forum and 88 U.S. eighth-grade essays from the Hewlett Foundation’s ASAP dataset.
What did the research find?
The performance of these detectors on students who spoke English as a second language was, to put it in terms no good teacher would ever use in their feedback to a student, atrocious.
The AI detectors incorrectly labeled more than half of the TOEFL essays as “AI-generated” with an average false-positive rate of 61.3%. While none of the detectors did a good job correctly identifying the TOEFL essays as human-written, there was a great deal of variation. The study notes: “All detectors unanimously identified 19.8% of the human-written TOEFL essays as AI-authored, and at least one detector flagged 97.8% of TOEFL essays as AI-generated.”
The detectors did much better with those who spoke English as their first language but were still far from perfect. “On 8th grade essays written by students in the U.S., the false positive rate of most detectors is less than 10%,” Zou says.
Why are AI detectors more likely to incorrectly label writing from non-native English speakers as AI-written?
Most AI detectors attempt to differentiate between human- and AI-written text by assessing a sentence’s perplexity, which Zou and his co-authors define as “a measure of how ‘surprised’ or ‘confused’ a generative language model is when trying to guess the next word in a sentence.”
The higher the perplexity and more surprising text is, the more likely it was written by a human, at least in theory. This theory, the study authors conclude, seems to break down somewhat when evaluating writing from non-native English speakers who generally “use a more limited range of linguistic expressions.”
What are its implications for educators?
The research suggests AI detectors are not ready for prime time, especially given the way these platforms inequitably flag content as AI written, and could potentially exacerbate existing biases against non-native English-speaking students.
“I think educators should be very cautious about using current AI detectors given its limitations and biases,” Zou says. “There are ways to improve AI detectors. However, it’s a challenging arms race because the large language models are also becoming more powerful and flexible to emulate different human writing styles.”
In the meantime, Zou advises educators to take other steps to try and prevent the use of AI to cheat by students. “One approach is to teach students how to use AI responsibly,” he says. “More in-person discussions and assessments could also help.”
Tech & Learning Newsletter (https://www.techlearning.com/news/ai-detectors-discriminate-against-non-native-speakers-says-stanford-research)
SIMILAR STORIES OF “A.I.” FAILURES ARE EVERYWHERE; INCLUDING THE “CITY OF SAN FRANCISCO” AND “GM”…CONT…The California Department of Motor Vehicles has suspended Cruise from operating its driverless taxis in San Francisco, effective immediately, for allegedly withholding footage of a severe crash involving one of the company’s robotaxis.Oct 24, 2023 (https://www.sfchronicle.com/bayarea/article/s-f-cruise-driverless-taxis-suspended-18445110.php)
California DMV suspends Cruise’s self-driving car permits, effective immediately
Published Tue, Oct 24 20231:55 PM EDTUpdated Tue, Oct 24 20234:29 PM EDT
Hayden Field@haydenfield (https://www.cnbc.com/2023/10/24/california-dmv-suspends-cruises-self-driving-car-permits.html)
The California Department of Motor Vehicles on Tuesday suspended Cruise’s deployment and testing permits for its autonomous vehicles, effective immediately.
The news follows a barrage of safety concerns and incidents since Cruise, owned by General Motors, received approval in August for round-the-clock robotaxi service in San Francisco.
“When there is an unreasonable risk to public safety, the DMV can immediately suspend or revoke permits,” the California DMV said in a statement….CONT…
GM Lost A 10-Year Battle With Tesla, Pulling The Plug On A Long Line Of EVs
Contributor
I cover and review electric vehicles and consumer tech.
My Chevy Bolt in the Mojave Desert near the town of Mojave, CA.Credit: Brooke Crothers
Despite what GM executives may say, it is not leading in electric cars.
GM does, however, have the dubious distinction of being a leader in killing off EVs. That hall of shameful decisions includes killing the EV1, the Chevy Spark EV, the Cadillac ELR, the Chevy Volt, and the Chevy Bolt. I’ve owned two of those EVs.
Who killed the (GM) electric car?
General Motors beat Tesla to market with an EV by more than a decade. The GM EV1 debuted in 1996, twelve years before Tesla’s first car, the Roadster.
The EV1 was a remarkable car, way ahead of its time. And it instantly became a favorite of Hollywood celebrities, environmentalists and green energy advocates in the U.S….CONT…
WE CERTAINLY DO “NOT” NEED/NOR WANT TO KEEP HEADING TOWARDS “TERMINATOR/SKY NET/THE MATRIX” FOR THE EARTH,
HEEYY EVERYONE=”THE SUCCESSFUL EV’S, DO “NOT” NEED TO BE “SELF DRIVING”, BY “A.I.” IN ORDER TO HELP WITH RIDING THE EARTH OF ENVIRONMENTAL POISONING EMISSIONS!!!!!!!!!!!!
EV’S ARE JUST 1/ONE VERY NEEDED TRAVEL/TRANSPORTATION SOLUTION, WHAT ABOUT USING “TAP WATER” AND/OR SEA WATER AND ALGAE, AND “PEA NUT OIL” AND “COOKING OILS”=THEY ALL WORK AND “HELP THE EARTH” AND OUR AIR, SOILS, AND WATERS!!!
https://en.wikipedia.org/wiki/Stanley_Meyer%27s_water_fuel_cell
Local Students Build Car That Runs On Used Oil From Philadelphia Eatery
January 30, 2015 / 4:00 AM EST / CBS Philadelphia
“We get our vegetable oil from Federal Donuts. And we convert it to biodiesel,” says auto tech teacher Andre Brooks, whose students at West Philadelphia’s Workshop School built the car. The roadster, painted General Motors Ocean Blue, is an alternative-energy car.Jan 30, 2015
By Mike DeNardo
PHILADELPHIA (CBS) – One of the cars on display at the Philadelphia Auto Show is built by local school students — and it runs on used fryer oil from a local restaurant.
“We get our vegetable oil from Federal Donuts. And we convert it to biodiesel,” says auto tech teacher Andre Brooks, whose students at West Philadelphia’s Workshop School built the car.
The roadster, painted General Motors Ocean Blue, is an alternative-energy car.
“This is going to be the only car there this color, and the only car that’s going to be able to say that a bunch of kids did it,” says teacher Mike Lumb.
Lumb says it literally comes in almost 500 different boxes, “and we have to put the whole body together.”
“We have a track record of building cool cars that are earth-friendly,” adds principal Simon Hauger.
Hauger says the car will compete in April’s Green Grand Prix:
“We’re expecting about 98 miles per gallon on this vehicle.”
Senior Joshua Pigford got to meet President Obama last summer when Workshop students were invited to the White House.
“He wanted to drive it,” Pigford says. “He really liked it, but the Secret Service wouldn’t let him drive the car.”
The Workshop team will compete against college and private teams.
