どころか vs ばかりか — both feel like 'not only... but'?

「彼は日本語が上手どころか、英語もペラペラだ」and 「彼は日本語が上手なばかりか、英語もペラペラだ」both seem to translate as 'not only is he good at Japanese, he's also fluent in English.' Is there actually a meaningful difference or are these interchangeable in casual use?
Post
沐學 Mizuru