The mother of the children, Lori Doppelheuer, 35, had escaped the fire but went back into the house to try to rescue her three children, Christopher Kearney, 10; Eagan Hargis, 3; and Kieran Hargis, 1, Maysville Mayor David Cartmell said. All four died.
The other victim was Larry Brickels, 68, who was not related to the family, Cartmell said.
The fire damaged five of 11 homes that are part of a set of rowhouses dating back to 1840, Cartmell said. Some of the houses may need to be razed, he added.
Maysville, a city of about 9,000 residents, sits on the Ohio River about 110 miles northeast of Louisville.
Authorities, including an arson unit from the Kentucky state police, are investigating the cause of the fire, Cartmell said.