Yes, we could easily forget our native language. However, I don’t want to forget the Japanese language. If I should lose my native language, I would be a nothing. I am learning English and German as second languages.
For a children it may be possible as some people said, that just happened with Steve as he tells in his book, that he had no recollection of his Swedish after he went to Canada at the age of 5 and learned English only.
For an adult I guess it’s nearly impossible (maybe exists some exceptions), but indeed the language would become rusty if no use at all, as any other language that the person might speak if it’s not used often