ChatGPT 3.5 and ChatGPT 4 exceeded the proficiency threshold for questions on the AAOS shoulder-elbow examination. ChatGPT may not be accurate enough to replace clinical decision-making.