

FPGA Stuff
My first experience with programmable devices was the PAL22V10. It made a good address decoder and could do simple logic. Later, I used CPLDs as peripherals to do all the fast real time stuff CPU's were not capable of, like fast counting and timing, and state machine based processing. A few years ago I got into FPGAs with the Xilinx Spartan 3E and 3A parts. It changed my world when I realized that I could create complete self-contained systems with a built-in CPUs, communications, signal processing, and fast control capability.
With PALs and CPLDs, using languages like ABLE or CuPL, or using schematics made sense. Once you pass a certain level of complexity, however, you need to move to a higher level of abstraction. This is where VHDL and Verilog come in. After working with both, I tend to prefer VHDL as it moves me a little further up from the hardware. It fits well with my many years of firmware development, where I gradually learned to trust the compiler. If you come from a purely hardware background, however, then Verilog, with variable types like "wires" and "registers," allows you to at least feel closer to the hardware.
Once nice thing about FPGAs as compared with CPU firmware - When marketing asks for some new feature after you are well into a project, you don't have to worry about the new function slowing down the code you already wrote. It all runs in parallel !
And there have been big improvements in just the past few years. The Xilinx Spartan 3 parts have 90nm geometry, but Xilinx has since gone down to 45nm with the Spartan 6, and has already announced the Artix and Kintex which will be 28nm.
With parts in the $10 range, and tools that are mostly free, it's easy to try FPGAs and find out their power and flexbility for yourself.
CPU Soft Cores
With the larger fabric FPGAs, if you are so inclined, it is possible to include one or more soft core CPUs. Xilinx offers the 32-bit MicroBlaze for a reasonable license fee, and offers the 8-bit Picoblaze at no cost. And there are quite a few third party processors. The open source opencores website has a number of free CPU cores, such as the AVR that allow you to use industry standard C/C++ compilers.
Having done a design with MicroBlaze, I would say that the advantages of using a soft core CPU are about evenly offset with various disadvantages. While it might seem like it's free to put a CPU into the fabric, there is a substantial learning curve with complex tools such as Xilinx Platform Studio, and you must use a larger more costly FPGA then you could have used without the CPU. You also have to study large and complex datasheets for the peripheral IP Cores to figure out all the registers and settings. For example, the datasheet for the Xilinx SPI interface is 38 pages long !
The performance of the MicroBlaze CPU is pretty much the same as ARM7 CPUs, as long as you enable cache in the MicroBlaze. Remember that you are executing out of external RAM. The EEMBC CoreMark website shows benchmark numbers for various ARM CPUs and for the MicroBlaze as well.
Another option is a hard core CPU such as the Cortex M3 in the Microsemi Actel SmartFusion FPGA. Hard CPU cores generally run faster and use less power as compared with soft CPU cores. These parts include A/D converters and programmable analog functions that allow creation of a complete data acquisition and processing system. The downside as you might expect, is that the FPGAs cost is higher.
Are CPUs Even Necessary?
And finally, another option is to simply challenge the paradigm of needing to use a CPU for every sequential processing need that arises. Think in terms of doing more in parallel, and doing it without a CPU. Even things that are serial in nature, such as parsing an arriving message, can be done without a CPU. I realized this after I created a Modbus message handler in straight VHDL with relative ease. CPU's are overrated !