IOMMU Client设备DMA配置过程分析(九)
1.设备树
cp0_pcie0是一个PCIe RC控制器,使用SMMU将PCIe设备的IOVA转换成物理地址,使用iommu-map-mask
和iommu-map
定义PCIe设备使用的Stream ID。设备树定义如下所示。
[arch/arm64/boot/dts/marvell/armada-ap80x.dtsi]
smmu: iommu@100000 {compatible = "marvell,ap806-smmu-500", "arm,mmu-500";reg = <0x100000 0x100000>;dma-coherent;#iommu-cells = <1>;#global-interrupts = <1>;interrupts = <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,<GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,<GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,<GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,<GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,<GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,<GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,<GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,<GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>;status = "disabled";
};[arch/arm64/boot/dts/marvell]
&cp0_pcie0 {// iommu-map四个元素分别为BDF、引用的SMMUphandle_node、Base StreamID、数量iommu-map =<0x0 &smmu 0x480 0x20>,<0x100 &smmu 0x4a0 0x20>,<0x200 &smmu 0x4c0 0x20>;iommu-map-mask = <0x031f>;
};
2.dma_configure
设备总系类型数据结构bus_type中定义了dma_configure和dma_cleanup两个回调函数,前者用于建立该总线上设备的DMA配置,后者用于清理该总线上设备的DMA配置。
[include/linux/device/bus.h]
struct bus_type {......int (*dma_configure)(struct device *dev);void (*dma_cleanup)(struct device *dev);......
};
当设备或者驱动初始化的时候都会调用到really_probe函数。若设备所属总线定义了dma_configure函数,则会调用该回调函数建立设备的DMA配置。
[drivers/pci/pci-driver.c]
const struct bus_type pci_bus_type = {.name = "pci",.match = pci_bus_match,.uevent = pci_uevent,.probe = pci_device_probe,.remove = pci_device_remove,.shutdown = pci_device_shutdown,.dev_groups = pci_dev_groups,.bus_groups = pci_bus_groups,.drv_groups = pci_drv_groups,.pm = PCI_PM_OPS_PTR,.num_vf = pci_bus_num_vf,.dma_configure = pci_dma_configure,.dma_cleanup = pci_dma_cleanup,
};
[drivers/base/platform.c]
const struct bus_type platform_bus_type = {.name = "platform",.dev_groups = platform_dev_groups,.match = platform_match,.uevent = platform_uevent,.probe = platform_probe,.remove = platform_remove,.shutdown = platform_shutdown,.dma_configure = platform_dma_configure,.dma_cleanup = platform_dma_cleanup,.pm = &platform_dev_pm_ops,
};
3. DMA配置过程
下面以PCIe设备为例,介绍PCIe设备初始化的时候,DMA配置流程。具体工作如下:
platform_bus_type
定义了pci_dma_configure
函数,因此在really_probe
函数里面会调用该函数设置DMA相关配置。- 首先解析
"dma-range"
属性,"dma-range"
属性定义了PCI地址到CPU地址的转换关系(inbound memory)。 - 使能ACS。
- 解析
"iommu-map"
和"iommu-map-mask"
属性,然后将PCIe设备的BDF转换成对应的StreamID。 - 调用SMMU驱动提供的
arm_smmu_of_xlate
函数,将转换后的StreamID保存到iommu_fwspec
数据结构中。 - 调用
iommu_probe_device
初始化设备,这部分内容在ARM SMMUv3控制器注册过程分析(八)介绍过,这里不多赘述。
of_map_id
函数首先解析"iommu-map"
和"iommu-map-mask"
属性,然后将设备的BDF转换成StreamID。流程如代码所示。
int of_map_id(struct device_node *np, u32 id,const char *map_name, const char *map_mask_name,struct device_node **target, u32 *id_out)
{// 读取"iommu-map"属性map = of_get_property(np, map_name, &map_len);....../* The default is to select all bits. */map_mask = 0xffffffff;// 读取"iommu-map-mask"属性if (map_mask_name)of_property_read_u32(np, map_mask_name, &map_mask);// iommu-map-mask & BDF,通常情况下,只屏蔽function三位masked_id = map_mask & id;// 遍历所有iommu-map定义的数据for ( ; map_len > 0; map_len -= 4 * sizeof(*map), map += 4) {struct device_node *phandle_node;u32 id_base = be32_to_cpup(map + 0); // 解析BDFu32 phandle = be32_to_cpup(map + 1); // 解析引用的SMMU的phandle_nodeu32 out_base = be32_to_cpup(map + 2); // Base StreamIDu32 id_len = be32_to_cpup(map + 3); // 长度......// *id_ou = iommu-map-mask & BDF - BDF + Base StreamIDif (id_out)*id_out = masked_id - id_base + out_base;return 0;}/* Bypasses translation */if (id_out)*id_out = id;return 0;
}
参考资料
- linux 6.12.35 source code.