You are not missing anything. What you do depends entirely on your situation. However, consider this:
It is very common to do parameter validation in a setter. For example, let's say I have a class with field that can hold a value 0 through 10 (the "throws" is unnecessary for the exception type below but I include it for clarity):
public class Example {
private int value;
public Example () {
}
public final int getValue () {
return value;
}
public final void setValue (int value) throws IllegalArgumentException {
if (value < 0 || value > 10)
throw new IllegalArgumentException("Value is out of range.");
}
}
Here, setValue() validates 'value' to make sure it sticks to the rules. We have an invariant that states "an Example will not exist with an out of range value". Now let's say we want to make a constructor that takes a value. You might do this:
public class Example {
...
public Example (int value) {
this.value = value;
}
...
}
As you can see, there is a problem. The statement new Example(11) would succeed, and now an Example exists that breaks our rules. However, if we use the setter in the constructor, we can conveniently add all parameter validation to the constructor as well:
public class Example {
...
public Example (int value) throws IllegalArgumentException {
setValue(value); // throws if out of range
}
...
}
So there are many benefits to this.
Now, there are still cases when you might want to assign values directly. For one, maybe you don't have setters available (although I would argue that creating private or package private setters is still desirable, for the reasons mentioned above, for reflection/bean support if necessary, and for ease of validation in more complex code).
Another reason might be that perhaps you have a constructor that knows, somehow, ahead of time that valid values will be assigned, and therefore doesn't need validation and can assign variables directly. This is usually not a compelling reason to skip using setters though.
However, all-in-all, it's generally a good idea to use the setters everywhere when possible, it will usually lead to cleaner and clearer code that is easier to maintain as complexity increases.
Most of the examples you see where people set variables directly are just people being "lazy" - which is perfectly acceptable if the situation warrants it (perhaps you're writing a quick test program or application and don't want to implement a bunch of setters, for example). There's nothing wrong with that as long as you keep the big picture in mind and only be "lazy" when it's appropriate.
Something I'd like to add based on some of the other answers here: If you override a setter in a subclass, and the data you are setting breaks invariants that the base class assumes, then either the relevant setters should be made final or the base class should not make those assumptions. If overriding setters breaks base class invariants then there is a bigger issue at hand.
You'll notice the getter/setter is final in the above example. This is because our rule is that "any Example must have a value from 0 to 10". This rule therefore extends to subclasses. If we did not have that rule and if an Example could take on any value, then we would not need a final setter and could allow subclasses to override.
Hope that helps.