Pictured below is how I diagram a household with age and gender divisions in the early census. In 1820 Henry and Jeremiah lived on the Jackson County side of the Hurricane Creek area while John and Jesse lived on the Smith County side of the Hurricane Creek area.

The Head of Household name goes in the center with his or her age category beneath. Unnamed male residents go to the left side and unnamed female residents go to the right side, in broad age divisions. 
Exactly how old are these? What are their names? You will never get more than a good guess from an early census record. You need other documentation to prove anything more than a name of the Head of Household, number of residents, and the County of Residence in the Census year.

                         Age divisions below are: 0/10 - 10/16 - 16/26 - 26/45 - 45+ 
Except for the Head of Household, the early census doesn't tell us who occupies the home.
                                            (Excuse my crude artwork).

Below is a 1940's family in a 1820's style census. This is an example of how quickly drastic changes can occur and how hidden those changes are to future generations.  In 1940 this looks like a simple household. Let's assume this is a dad, mom, and four sons under 10.
Below: two years later. This is the same 1940 Head of Household. The only part that resembles the 1940 household (above) is the man and the age of his wife. Is this the same wife? Where are the other three children?
Below: in two more years there is no child with that same 1940 Head of Household and his wife. Is this the same wife as in 1940 and 1942?  The age groups are the same.

The example above is my dad's family. It would seem that the wife in 1940 is the same wife in 1942 and in 1944 according to the ages. Is it the same wife?  How could this possibly be the same family? How did his household lose three children in two years and then lose the fourth child in two more years? 
What happened to all four children in just four years?  

This is a real-life example of how little we learn about a family by simply looking at the diagram of a household, made on a particular day, in one particular year.  

Above is the outline of the Hurricane Creek Dickens families as they were on the day of the Census in 1820. It shows us very little of their yesterdays and nothing of their tomorrows. 

In our era, we expect all documents to be complete and precise. But even today, drastic changes can occur within a family that are not easy to explain without supporting documents.

On the diagram below, we can see that in four years my dad's entire family underwent drastic changes, not once but twice. Only the Head of Household name and the female ages remained the same. If this were an 1815 to 1820 household, no one could figure this out 150 - 200 years later. Even if names and ages were given, a 1940 to 1945 household would take time and research to figure it out and all the details would never be known.
I'm the only living person who knows the details of this household and no one else now living ever knew of wife #2, Martha (until now).

1940 to 1942:
Divorce from Carrie. Dad married Martha.
Child 1 & 2 went to live with grandma in another state.
Child 3 stayed with dad.
Child 4 stayed with mother Carrie.

1942 to 1944.
Divorce from Martha. Remaining child (#3)went to live with aunt in another state.
Dad then married Viola.
  1944 on. Dad stayed with Viola with no children.
Everything  in this family changed from 1940 to 1944 except the Head of Household name and the
parents ages. 
In this diagram:
Child #1 is 8 years old (me).
Child #2 is 4 years old.
Child #3 is 3 years old.
Child #4 is 1 year old.
In the year, 2007, we had Presidential Candidates who have three living wives (divorces applied). If we were to diagram their families in the 1820's format, they would look as strange as the family above. So would mine. And maybe yours?

Do the shortcomings of the 1820 Census give us the right to modify family structures wherever we please? No.
When the census is silent we have to rely on other sources to establish relationships. But other sources are not a persons guessing, intuition, hunches, wishful thinking, or copying from some other researchers unsupported claims. 
We have to look for sources such as oral history and family traditions that have been preserved and passed down through the generations, personal journals, family Bibles or books that have personal information written in, and whatever public records that exist.

You can see from the above example that without names and ages, it is difficult to analyze a family. And if you could correctly analyze the family as it was on the day of the census, you could not account for changes that may occur within the family beginning on the day after the census information was recorded. 
An explanation of the early census is not necessary for people who are accustomed to doing research but there are several million folks who don't understand. This is a quick and simple explanation to show
why there can be uncertainty and controversy when using the early census to establish relationships. 

How does an early census look? There are many volunteers who patiently go through old microfilms and present to us the clean, neatly typed names and numbers that researchers find on the Internet. What you then see is a drastic improvement over the original. Below is part of a page from the original 1820 Smith county, TN Census. This is representative of what the volunteers work with. It's slow, tedious work. It's out of pages like this that diagrams such as I have above, are constructed. John Dickens' household is on Line 533 (6th line from the top). 

The original that volunteers work with is clearer than my reproduction here but you can get an idea of how much we should appreciate the volunteers.

Now a for instance: (still using the 1820 format and conditions) if the first diagram was dad's actual household in 1939 (imagine 1819) or before, when there was no census taken, and the second or third diagram was of a census day in 1940, (imagine 1820) we would never know that those children in 1939 (imagine 1819) all belonged to dad and we wouldn't know who the mother was without doing a heck of a lot of research and even then we might not ever figure it out. 
Let me be quick to add, the early census records are often the only source of information we have. The census is usually the first place we look for traces of our families. Thank goodness they exist. Without them many of us would be totally in the dark when it comes to finding our forefathers.  
                           Dickens of Hurricane Creek
Including: Maggart, Sullivans Bend, Elmwood, Chestnut Mound, Smith County, TN
Page 07.

A look at the 1820 Census format.

Page 07.
This illustration is repeated in other sections.