Well, it depends. Those kinds of subscribers you just mentioned are "dead-subscribers" which means they just subscribe to your channel, but do not watch your videos. The results are the same if you do "Sub-4-Sub" most of the time, where even if your sub count increased by 1, your active viewers remain the same. Yes, they are very annoying as even if they increase your sub count by one, you don't know if they'll be watching your videos. Anyways, there are 2 parts that I can think of for the question.

In 1 side of the story, do subscribers matter to a Youtuber? You and I both know the answer, YES! You can build a community with the active subscribers and continue growing it as more people join in. They are there to support you in what you do by watching your videos, providing feedback or comment and so on. They are one of the most important people for a Youtuber.

On the other hand, do subscribers matter when checking out channels? Absolutely not! There are a lot of really good channels out there that no one even knows exist! (You can even check out some of the entries in the Shout Out Competition! A lot of them are very talented in their work). And also, there are a lot of channels that has a high amount of subscribers but they either do not upload anything (Like the "100,000 Subscriber with No Videos" people) or upload a video where no one is interested in watching it. There are also people who actually buys subscribers for their channel (Yes, these people exists). Which is why whenever I want to check out someone's channel, I don't judge the channel by their sub count, but I judge them by other things like consistency, audience engaging, keeping the audience's attention in a video and so on.

So, it all depends on what the question is directing to, whether subscribers matter to a Youtuber or subscribers matter when looking for new channels.

I think subscribers are important in any regard, though It's really the ones who genuinely sub because they enjoy coming back to see your next video that really help foster growth in your channel community. At the end of the day, each member subscribed does mean one extra feed out there in the world that displays your new content, whether it's paid attention to or not, as opposed to someone not being subscribed at all and missing any chance at all of seeing your content entirely. The sub 4 sub trend is a hollow effort to raise a number that in such a context is meaningless. Ultimately, quality > quantity. The more active subscribers you have that engage with your videos the further the fruits of your labor will be heightened and eventually with time and consistency this leads to more subscribers and a community snowball effect.