A YouGov online survey of 1,015 parents found only half of children with speech problems received expert help.
The survey, for England's first "Communication Champion" Jean Gross, found some three-year-olds were unable to say a single word.
Ms Gross said the proportion of children with problems is "high" and that getting help early was essential.
"Our ability to communicate is fundamental and underpins everything else. Learning to talk is one of the most important skills a child can master in the 21st Century," she said.
"The proportion of children who have difficulty learning to talk and understand speech is high, particularly among boys.
"It is essential that all children get the help they need from skilled professionals as early as possible."
Six out of 10 people questioned for the survey said the ability to talk, listen and understand was the most important skill for children to develop in the early years.
This priority came ahead of the ability to interact with others (26%), reading skills (11%), numeracy skills (2%) and writing skills (1%).
All those questioned said they looked at picture books with their children, told them stories and sang nursery rhymes with them - all activities which boost language development.
The survey showed that the majority of children (51%) did not enjoy looking at picture books with their parents until they were over six months old, but 18% enjoyed this at three months or younger.
Children from more affluent families were reported to enjoy looking at picture books, and listening to stories and rhymes, at a younger age than children from less affluent families, researchers said.
The most common age for children to say their first word, according to the parents surveyed, was between 10 and 11 months.
More girls than boys (34% against 27%) said their first word before they reached nine months. But 4% of children had not said their first word by the age of three.
There were no real social class differences in when children said their first word, the researchers said.
Most parents (95%) could remember what their child's first word was.
Among those surveyed, that word was most likely to be "Dadda" (15%) or "Daddy" (13%), with "Mama" (10%) and "Mummy" (8%) trailing a little behind.
Speech experts generally think the "da" sound is easier for babies to say than "m".
The YouGov research involved 1,015 parents of children aged one to to seven, questioned online in December. The figures have been weighted to provide a representative sample.