Infrastructure exists to allow the economy to work. It can be physical (such as roads, ports, rail) or it can be metaphysical (legal, education levels etc). It can be privately funded (such as a rail line to a mine), or funded through government spending (such as roads).
Government spending on infrastructure is all about the provision of public goods. Pure public goods are essentially those where you cannot exclude someone from using it, and where one person's use does not prevent another person from using it. Public goods are not supplied by private market actors because it is difficult to generate revenues from their use. It is not that people don't want to have them, or are not willing to pay for them, but rather that it is not practical to charge people for their use. A common example is spending on a road. We all want the road, and we are all willing to pay for the road, but it is not possible for a business to come and build the road and to charge people to use it (although toll roads can support private investment for some road projects).
Some public goods can enable other parts of the economy to work at their full potential. For example a road linking a town to a major city can allow that town to participate in the broader economy. In this regard infrastructure spending may be linked to what is typically called 'capacity constraints'. Bottle necks can limit economic activity to only be as good as the slowest moving part so to speak (even where there may be sufficient capacity). Where these bottle necks can be addressed by infrastructure that could be classified as a public good, then the Government has a clear motivation to provide it.
In practice however, many infrastructure projects are not pure public goods. For example rail networks can charge passengers a fee to recover costs. One reason why the government may decide to subsidize such infrastructure is because a part of the benefit is useful to everyone. For example, while only some people might use a train service, we all benefit from having it there. Some may benefit through fewer cars on the road (and therefore less traffic and less need for government spending on roads). Others may benefit merely from having the rail network in existence as a backup in case their car fails. Alternatively there may be an environmental efficiency dividend for everyone. Either way, part of the benefit cannot be paid for by the price paid by the consumers who use the service. This partial benefit that cannot be charged to the broader public is referred to as a positive externality. You may have heard of negative externalities in relation to pollution - here the concept is the same, however in the case of infrastructure spending we normally think of these externalities as being good for us.
The fiscal benefits of infrastructure spending was mentioned above. This is true, but it is better to capture this line of thought when thinking about fiscal policy, as it is not unique to infrastructure spending. The unique benefit associated with infrastructure spending is that it enhances welfare in the ways described above.