[…] Would you dismiss your doctor for their “appeal to authority” when they open a medical textbook? […]

Trusting the doctor’s word simply because they are a doctor would be an appeal to authority; whereas, referencing a medical textbook would be citing a source, and therefore not conjecture.