[pci] Force completion of ECAM configuration space writes

The PCIe specification requires that "processor and host bridge
implementations must ensure that a method exists for the software to
determine when the write using the ECAM is completed by the completer"
but does not specify any particular method to be used.  Some platforms
might treat writes to the ECAM region as non-posted, others might
require reading back from a dedicated (and implementation-specific)
completion register to determine when the configuration space write
has completed.

Since PCI configuration space writes will never be used for any
performance-critical datapath operations (on any sane hardware), a
simple and platform-independent solution is to always read back from
the written register in order to guarantee that the write must have
completed.  This is safe to do, since the PCIe specification defines a
limited set of configuration register types, none of which have read
side effects.

Signed-off-by: Michael Brown <mcb30@ipxe.org>

1 files changed, 10 insertions, 1 deletions

diff --git a/src/drivers/bus/ecam.c b/src/drivers/bus/ecam.c
index 1d57bd2a0..376f34957 100644
--- a/src/drivers/bus/ecam.c
+++ b/src/drivers/bus/ecam.c
@@ -235,7 +235,7 @@ int ecam_write ( struct pci_device *pci, unsigned int location,
 	if ( ( rc = ecam_access ( pci ) ) != 0 )
 		return rc;
 
-	/* Read from address */
+	/* Write to address */
 	index = ( pci->busdevfn - ecam.range.start );
 	addr = ( ecam.regs + ( index * ECAM_SIZE ) + where );
 	switch ( len ) {
@@ -252,6 +252,15 @@ int ecam_write ( struct pci_device *pci, unsigned int location,
 		assert ( 0 );
 	}
 
+	/* Read from address, to guarantee completion of the write
+	 *
+	 * PCIe configuration space registers may not have read side
+	 * effects.  Reading back is therefore always safe to do, and
+	 * guarantees that the write has reached the device.
+	 */
+	mb();
+	ecam_read ( pci, location, &value );
+
 	return 0;
 }