The possible solution to your problem is a rearranging of your data.
For this, you need three tables. The users, the groups, and a 'link'-table.
users:
user_id
userdata1
groups:
group_id
groupdata1
link
user_id
group_id
In the link list, you create one dataset for each connection.
So, if user 1 is in groups 2 and 3, the link table will have two entries with the user_id 1. One linking to group_id 2, and one linking to group_id 3.
To get back to your example, the query to get all users from group 2 is:
SELECT * FROM users, link WHERE link.user_id = users.user_id AND (group_id = 2 OR group_id = 3);
This will output all the users, who are in one of the groups, some as duplicates (those, who are members of both groups). If you want to avoid duplicates, add 'GROUP BY user_id'