I'll try to answer your question as best I can from my 3rd year core experiences (notably 80% outpatient, 20% inpatient, midlevels on every rotation). To fully answer the original question, most patients initially trust everyone they encounter in healthcare (or barring midlevel attitude problems are usually too modest to say otherwise), and that is the problem--what they don't know about medical training can actually hurt them. Most pay for and expect a service, not a relationship.
Outpatient medicine is completely based on the rapport the midlevel or physician builds. Patients are free to switch, but with the level of continuity and trust in those settings it doesn't make sense to do so. Certain specialties are much more sensitive to this rapport, such as Ob. For example, I have seen patients actively seeking the care of nurse midwives only, and many of those patients have devoted trust in seeing only the midwife in the practice after multiple births. It wouldn't make sense for someone who has been delivered 5 times by a midwife to suddenly transfer her care over to a physician who doesn't know anything about her body's history. The relationship between physicians and midlevels in Obgyn is unique and fascinating...some would argue it is necessary. In generalist outpatient specialties (FM/Peds/IM) the relationship doesn't make any sense to me...an extender of a physician generalist shouldn't exist and makes no sense on paper. How do you extend the services of a generalist? Even if you're talking about a midlevel that provides limited services that the physician really isn't comfortable providing but can if necessary (like a well-visit ob midlevel that does speculum exams all day), the patient would almost certainly be better off with a midlevel at a specialist practice. The docs training midlevels to be full spectrum generalists/specialists are doing a disservice to their specialty.
Inpatient medicine is team based. Leashes are typically kept very short within those teams, but yes the inpatient PA/NPs I've seen have good rapport with patients, and they're treated like physicians by patients. When patients only want to see the physician it can be very awkward, and sometimes the physicians have stepped in to bolster the patient's trust in their nurses or midlevels (you have to remember that these people are often trained BY the physicians, and so are valued as true physician extenders--tools that allow physicians to be in more places). The level of procedural capability and diagnostic skill of some of the better-trained and institutionalized inpatient midlevels has baffled me. In my experience, midlevels are actively reaching more areas of inpatient medicine and surgery than people here will give them credit.
After a while of seeing this in just about every rotation it started to make sense to me: Inpatient medicine is moving to be much more team based, and trust and responsibility has to be placed on the system rather than individuals. Much like a sports team, physicians are often the team leaders or coaches covering multiple functions and ensuring their players are performing to spec. Yes, they can also step in and play the game themselves, but it's often not time-efficient to do so. Instead what I have noticed is that midlevels as true physician extenders are often really great at a few dozen things and provide those services at higher quality than any physician just from having done them so often. Outpatient medicine can be concentrated and focused, or it can be a miniature team. Patient concerns can vary dramatically from cost, ease of access, rapport, and other factors. With these varying concerns you can end up shooting craps with who is taking care of you.
Knowing what I know now, I do not question the role of inpatient midlevels, but I seriously question the role of anyone outpatient who hasn't had formal training in a residency. I do sense that some midlevels have the ability to be better than physicians at some things, but the picture of their entire skillset and knowledge base is tiny by comparison. Even more, I question how qualified that person is to manage the entirety of my care on the spectrum of low-high acuity. In a sense, outpatient medicine is the wild west and midlevel leash length varies dramatically across practices. However, in my experience, midlevels necessarily function better as cogs in a larger system than they do as directors of individualized care, and are better suited for the controlled inpatient medicine and surgery environment as trained technicians.