Most of the houses in Reykjavik are not more than 4-stories high. You will definitely find some exceptions ... but this is the standard supremum* :o). There are block of flats away from the center. In the rest of the area you will come across 2 to 3 stories little houses or new "Beverly Hills" areas built for the rich Icelanders on the outskirts.
You can tell that most of the houses in the center of Rvk are quite old, the coat of the houses is slowly falling down... you actually would not find that many reconstructed houses in this area. But there is definitely something catching about these wooden or stone houses, some of them actually even wear a name, so that you do not get lost... :o). And if by any chance you do... then nearly all the houses have different colours there - red, green, yellow, blue, pink... the streets thus seem much happier.
The houses are either detached... creating somewhat "villa" areas and each house has little garden around, bbq ready to put out in good weather and if lucky then even a hot pot for cold icelandic nights. Some of the houses are connected to each other, these are mainly visible in center along the main streets. Reykjavik is really a cozy little town.
* The supremum is the least upper bound of a set S, defined as a quantity M such that no member of the set exceeds M, but if is any positive quantity, however small, there is a member that exceeds (Jeffreys and Jeffreys 1988).