Eh, at the end of the day, they're different languages and are following different paths. Sure they're superficially similar, but that's not all to it.
Python and Ruby are pretty similar languages overall, but neither has "killed" the other. Why? Because despite their similiarities, they offer different things. I generally prefer to use Python over Ruby, no real reason for it, I just prefer the language. With Rust and Swift, there are some pretty major differences that can't be waved away as personal preference, Rust's borrow-checker isn't something Swift can integrate overnight (and wouldn't likely do) and Rust can't just grow some ARC equivalent (and almost certainly won't).
And hey, some of my favorite minor features for Rust are cribbed from Swift. Namely `if let` and `while let`. I'm not scared of a little competition, keeps us on our toes and makes sure we keep looking for ways to improve the language and ecosystem.
Just to note that by far and away, the most common cause of an ICE is a missing error message. Others are related to unstable features, which isn't particularly fair, since they're unstable for a reason. The remainder tend to be caused by odd combinations of features we didn't think of (which more often than not tend to be constructions with simple, working equivalents), often they become a question of "do we want to support this or not".
Trust me, there are far worse things lurking in the compiler than a few improperly-handled error cases.
An evolving language is one thing. Pre-1.0 was more like many stages of violent metamorphasis. Some changes to the language required changing almost every line of code in the tree. Normal technical debt is when you find an old approach doesn't work any more and you need to fix it. Much of rustc's technical debt is from things like features disappearing from underneath it, or things designed before the features that you would use now even existed.
Looking at rustc as a regular large Rust project is unfair. A new project can at least assume that the features you are using today will exist tomorrow and design with that in mind. Your designs may become obsolete due to new features, but you aren't going to have to pick between a hacky workaround and a redesign/rewrite of 1000's of lines of code.