There are some old radiator sizing formulas but they will most often produce wildly oversized systems.
If you're good and accurate with measurement, are quite familiar with the construction of your home and like doing "new" things, go to www.hvaccalc.com and download the one-time use program for a modest fee.
To know a bit of what you'll need: surface area of walls exposed to the outside; surface area of glass; surface area of ceilings & floors; surface area of partitions adjoining "unheated" or "less heated" areas; levels of insulation in all these areas; basic type of construction to include sheathing, doors, windows, etc. Much of the surface area can be calculated automatically given dimensions. Since spaces loose most of their heat through the ceiling, it is important to be EXTREMELY thorough with ceilings and a given room will often have MULTIPLE "types" of ceiling; i.e. a room under a finished attic (the eave space will be different even if unheated), a projecting bay window, stairwell or a "bump-out" addition.
An ACCURATE heat loss calculation will tell you how much heat the structure requires as a whole and how much each space requires as well.
Radiators are sized based on their Equivalent Direct Radiation or EDR. This is an OLD throwback to the earliest form of radiator that was essentially a flat panel looking something like a mattress. EDR is measured in square feet and sizing data is readily available for most old rads and all new rads.

Unless you're REALLY handy and are willing to study quite a lot, installing a good hydronic system isn't a DIY project. Even if you size everything and run your own piping have a pro install the boiler--or at an absolute minimum pay them well to review your boiler installation.
Once you've computed your heat loss, the next step is to determine the general temperature at which you want the system to run.
Remember that radiators DO radiate significantly and that radiation is generally regarded as the most comfortable and efficient method of space heating. In general, the lower the temperature of the radiator the higher its proportion of radiation; the higher the temperature the higher its proportion of convection.