You can share games with 3 other people ONLY if the game is bought digitally as in NO DISK.
Lets say I buy a game, 3 of my friends can download my Xbox profile onto their consoles and then download the game and play it. If a fourth person does this, then the first person who got the game shared will lose the ability to have it, if a fifth person does it then the second person will lose it and so on.
I have an Xbox One myself and I'm almost positive this is how it works. My cousins friend had titanfall and I was able to get it because he had the digital version. My cousin bought Destiny from gamestop so it was a disk version and I was NOT able to play the game. I could download it, but then it prompted me to insert the disk.