It is true that if your fetching of data is slowing your load times down, then lazy loading is a great solution. But applying it across the board seems to be a premature optimization. You would probably want to impliment it for each set of data and then test to see if it speeds up the app. If it does not, then it is not a good candidate for the lazy loading.
The way I have implemented the lazy loading caused no change in the data tier. It was all down in the business logic and presentation controllers. As I am not familiar with EJB I am going to assume that this will work for your java app. Any way, When I implement the lazy loading, I load no data (atleat none of the data I am going to load lazily), until it is needed. Then I call the data tier and get my data (or a subset of the data).
As for the connection concern, you will need to put checks in place to test the data connection to see if it is closed. That is, is you are pooling the data connections. Then if the connection is closed, then reopen it. But as with the actual lazy loading implementation, this should be done in your logic classes and not in the front end, so you don't have to duplicate this functionality many times.