It's a red flag regardless of the field its in (but would certainly be more awkward in your field of choice). The etiology does matter in the sense that if it's work ethic, interpersonal, or moral failure it's worse.
This wouldn't preclude the person that failred from getting a job of course but it will shut them out of many programs and competitive fields. If the failure is a result of a personal failing (absenteeism, lying, etc), then it can be very difficult to match.
Finally, generally speaking, program directors look at IM and surgery clerkship performance regardless of what speciality you apply for, as these are the two most challenging and longest core clerkships across the country.