A children's charity said it was "very disappointed" with plans to shut specialist hearing units at two schools in Torquay.