In 1915, residents of Duval County approved a one-million-dollar bond to build several new public schools. Construction started on Public School Number Four in 1917 and was completed in 1918 at a
cost of $250,000. The architect behind the project was Rutledge Holmes, who had relocated to Jacksonville after the Great Fire of 1901, and designed many of Jacksonville's famous landmarks.
The front facade of Public School Number Four is dominated by Doric Columns which support a neo-classic pediment portico. The classrooms were located on the 2nd floor which featured high ceilings
and large windows. The offices, library, auditorium, and cafeteria were all located on the 1st floor. A narrow set of passageways ran underneath the school to a boiler room, a room that is
unfortunately associated with many urban legends.
Later, the school would be renamed Annie Lytle Elementary after a longtime teacher and principal. The school building originally overlooked Riverside Park, but that would change with the
construction of the I-95 and I-10 interchange in downtown during the 1950's. Afterwards the school was left isolated and underneath the new interchange. In 1960, the last students would
walk through the hallways before the school was closed. Although there is no concrete evidence, there are suggestions that the building may have been used by Central Christian School in
the late 60's and early 70's. However, the city condemned the building in 1971 and used it for office and storage space for other schools.
Over the years, vandalism and arson have taken a toll on the building. In 1995 a fire destroyed the roof of the auditorium causing it to collapse into the building. Again in 2011 another
fire originated in the auditorium and caused minor roof damage.
In October 1999, Foundation Holding Inc. purchased the property with plans to demolish the existing and build "Lytle Place Condominiums" in its place. The proposed plan was denounced by
the public and surrounding historical societies. In response, the city designated the building a historic landmark in 2000.
Although the building is now a historic landmark, every year it still faces demolition. Today the school is known for its haunting legends. Because of these legends, the school has
earned the nickname "Devil's School" from some of the local residents.