Don't assume that your indoor olive plant is safer from pests than your outdoor tree. So, check them regularly and use proper prevention methods. It can outgrow its container, so repot as necessary.