At the ideal condition, when the circle has n points outside, n inside and 3 on it,
The distance from the center of exactly n points will be more than the radius, and exactly n points less than the radius.. and only 3 points equal to the radius.. This will help you see how many are outside/inside quickly at a particular instant..
Make a vornoi diagram on the 2D points, this helps in finding which point is near to which other points.. Search on it if you need to know the concept.
Start by making a circle from any 3 adjacent points..
store the distance of all points to the center of this circle in an array.. points with distance more than R are outside and distance less are inside.
Progressively, leave a point on the circle and choose a point nearest to the boundary of the circle but outside the circle, to make a new circle.. and recalculate the array.
If we call the count of points inside as Nin and count of outside as Nout..
Nout will start decreasing and Nin will start increasing slowly. Keep doing this till they become equal( ur done, if so) or Nin exceeds Nout..
If this happens, make a new circle by choosing a point inside and near the boundary.
Keep doing this until you get Nin == Nout at which point you have the solution..
Note that, taking the new point outside the circle, would increase the circle radius, and taking the new point inside the circle decreases the radius.
This is the first decent solution that came to my mind. This may need improving on certain aspects. I would expect the complexity to be much better than a brute force solution. I leave the calculation to you. :)