If you want to grow acid loving plants and you have an alkaline soil, the best way to lower the pH is to construct a raised bed and add lots of organic matter - manure and compost. Alternatively, if it's perhaps only one shrub, try growing it in a container using ericaceous compost.
Perennials
Species that require acid soils tend to be woodland plants such as Trilliums that enjoy the shade and a moist acid soil, and spring flowering small but beautifully formed Uvularias.
The bright red panicles of Astilbe 'Fanal' will light up any partially shaded area while Dicentra (Bleeding Heart) is another perennial that likes partial shade together with the mat forming Ajuga (Bugleweed). Lewisias are small, fully to frost hardy, perennials suitable for moist rock gardens as is the low spreading Phlox stolonifera (Creeping Phlox) and Oxalis acetosella (Wood Sorrel).
Other acid lovers are Pacific coast Irises that can vary in colour from yellow to purple and Meconopsis (Blue Poppy) that are fully hardy but need shade and a cool position. Foxgloves will grow in many conditions including acid soils.
There are heathers too many to mention and many types of ferns to choose from such as Matteuccia struthiopteris (Ostrich Fern) that likes shady and wet conditions. Osmunda regalis (Royal Fern) will tolerate sun but enjoys damper conditions than O. cinnamomera (Cinnamon Fern).
Shrubs
Rhododendrons, Azaleas and Camellias are the most well known and popular acid loving shrubs. There are many others that prefer or tolerate these conditions including Begonias and Hydrangeas, the latter whose flowers turn blue in acid conditions.
Fothergillas are deciduous low growing shrubs with sweet scented spikes of white flowers, their leaves turn yellow, orange and red during the autumn. They are closely related to Hamamelis (Witch Hazel) which has the same delightful characteristics. H. Mollis (Chinese Witch Hazel) shows off very fragrant yellow flowers in the winter.
Many of the shrubs require partial shade including Clethras like C. alnifolia (Sweet pepper-bush) which has scented white flowers, as does Styrax officinalis. Others include Leucothoe an evergreen or semi evergreen also with white flowers; Philesia magellanica has pink trumpet flowers and Desfontainia spinosa, an evergreen bush with holly like leaves and long tubular red flowers.
The long lasting fruits of the Pernettya will only result from their white flowers when there is a male and female plant present. 'Edward Balls' is undoubtedly male while 'Cherry Ripe' (red berries), 'Mulberry Wine' (magenta berries) and 'Wintertime' (white berries) are female.
Gardenias such as 'Fortuniana' with glossy leaves and fragrant double flowers are frost tender while the young shoots of the profusely flowering Pieris may be lost to a late frost. Stewartia malacodendron is frost hardy and bears rose shaped white and purple summer flowers
For the most delicate flowers choose an Andromeda, small evergreen shrubs with clusters of globular white and pink blooms or a Kalima such as K. latifolia (Calico Bush) that has distinctive cup shaped pink flowers in the late spring and early summer.
Climbers
Asteranthera ovata can be used as ground cover or a climber up to 12ft; Berberidopsis corallina is useful for a north facing wall and carries attractive pedant red flowers into the autumn; and Mitraria coccinea, a woody climber (6ft) with orange/red late spring flowers, is half hardy and should be grown in partial shade.
Trees
Quercus coccinea (Scarlet Oak) is a potential 70ft tree with a good autumn colour that prefers acidic soil, as does Amelanchier arborea, which also offers autumn red with small round fruits. Some Acers such as Rubrum or Red Maple produce their best colour in neutral to acid soils and most Magnolias prefer these conditions, while the scarlet berried Mountain Ash likes a soil that is only slightly acidic.