Yeah saw a bunch of threads on that. Can audit code, write scripts for trading based on Twitter posts of someone else.
Comical weaknesses but I prefer to focus on what it can do well so appreciate you pointing out the code writing.
I saw “write a 500 word summary of the French Revolution” and thought that was a taste of some places this is headed.
Largely I still think people are focusing too much on novelty at this point. I want “read these 10 published science studies and give me 3 possible future studies” or more generally, to be able to shovel it an incredible amount of information and have it digest it and distill down to the most salient or actionable concepts.
I want "read all studies on <any topic> find the commonalities and figure out why some of the results seemingly conflict with the others, explain the results and identify reasons for inconsistencies"
So many things work in some studies, don't work in others, but the studies aren't doing exactly the same thing.
And then a broader view that would find the things to do that on, sift through and rank all of the results.
I have a strong belief that as a species we've done the work to be much better at preventing and treating diseases, including things like cancer, Alzheimer's, etc, and even aging as a disease, but it's too difficult for any one person to sift through all of the collective data to find the underlying truths.
See this video where we found planets by both improving the physics and figuring out there was some bad data. I believe AI could help make progress on issues that are much more complex to sift through, but the AI has to actually understand the math and science well enough to figure what it all means. I'm not sure how close we are to being able to do that. I do know that some of what ChatGPT can do is really impressive, while some of it looks impressive, but is actually wrong