Nicolas I'm obviously not a tax lawyer and I don't know about France, so this is your responsibility to research...
But normally you only have to pay taxes if you receive interest. So you should go to the "My Money" menu and disable "I want interest". Then you won't get any interest and you should not have to pay any additional taxes or report anything to the tax office. Just make sure that you disable the interest.
Or if you enable it you probably have to report the amount to the tax office and pay a certain percentage to the French government. Your choice.