The proper way to hold the guitar is ultimately a matter of what is most comfortable to you but in the beginning there are some standard methods that you should follow.
If you are sitting down you should have a comfortable chair and preferably one with no arms, otherwise it will be difficult to get into a good playing position. Your back should remain straight but relaxed.
We are also going to discuss this from the perspective of a right handed player so if you are left handed simply do everything here in reverse.
The typical guitar is shaped so that there is an inward curve at the lower part of the body. When playing an acoustic guitar you will want to place that curve on the guitar body on your right thigh. The guitar neck should be fairly parallel to the floor and the headstock should extend to your left as you are sitting.
Your upper right arm should rest on the upper most part of the guitar body so that the bend of your arm comes down over the top corner of the guitar body. Your right forearm will snug against the front of the guitar and the angle of the forearm should be such that the palm of the right hand lightly covers the bridge of the guitar.
This right arm position should feel natural and not forced, and you should be able to comfortably balance the guitar in this position without help from the left hand. Your right forearm should be able to move and swing freely while keeping the guitar in position. We will see later how important this is.
The left hand should extend naturally down the length of the neck and be able to reach the nut without straining.
If you are playing electric guitar the best option is to use a guitar strap. The smaller body of an electric guitar can make it awkward when sitting.
Either way, holding the guitar will seem a little foreign to begin with but if it is extremely uncomfortable or a strain in any way you may need to tweak your positioning a bit until you find the right fit that works for you.