Support for SGEMM_DIRECT Kernel based on SME1

[vaiskv]

This PR contains support for sgemm_direct kernel based on SME1 architecture.
sgemm_direct kernel handles a special case of cblas_sgemm() level 3 API where aplha =1 and beta=0.

[martin-frbg]

Thanks. Convenient that this special case does not imply debugging TRMM and SYMM as with the general case SGEMM kernel in #5011 that I still hope to get to soon. :/
Having unguarded -march flags that require a recent compiler is not going to work for all users however, I think we'll need the same kludge as used on x86_64 - ifdef the source on HAVE_SME and use an empty function declaration in the #else branch if not available, then rely on the required compiler flags being provided by Makefile.arm64 (or kernel/Makefile in the DYNAMIC_ARCH case, as currently for SkylakeX and newer) . Also I don't think one can use HWCAP on Apple, there should be a feature flag HAVE_SME provided by the build scripts (and "eventually" by the runtime detection code in dynamic_arm64.c too)

[martin-frbg]

HarmonyOS doesn't seem to support HWCAP either, and AppleClang balks at the "else if" introduced in common_s.h. I'll see if I can unravel and test locally.

[vaiskv]

Sure, Thanks!. I am working on restructuring the code to HAVE_SME flag as per your suggestion.

[martin-frbg]

#elif ARCH_ARM64

[vaiskv]

Updated

[martin-frbg]

Embarassingly it looks as if I nuked the HAVE_SME I had put in cpuid_arm64.c back in november (#4971) with some later change... bad things still sometimes happen when I jump between machines that are not always on the internet :(

[vaiskv]

I have a question:
I am compiling the library with the following command:

make BINARY=64 CC=aarch64-linux-android33-clang ONLY_CBLAS=1 HOSTCC=gcc TARGET=ARMV9SME DYNAMIC_ARCH=1

From the interface/gemm.c file, SGEMM_DIRECT kernel gets compiled only when DYNAMIC_ARCH=1. So when the library is compiled with DYNAMIC_ARCH=1 and if the TARGET is set to ARMV8 instead of AMRV9SME, does the function defined in kernel/arm64/sgemm_direct_arm64_sme1.c also be part of the library? I am assuming it won't be part of it as we have guarded the file with HAVE_SME. But then how can we ensure the library is supported on all Arm targets (Arm v8 , v9 etc)?

[martin-frbg]

With DYNAMIC_ARCH, TARGET is only used for the common code (interface/gemm.c and all the other interfaces, driver/level3 and so on), and the codes under kernel/arm64 are compiled in a loop with TARGET_CORE set to each of the individual models ARMV8, ARMV8SVE, etc. supported in DYNAMIC_ARCH configuration. So what I tried to express is that your kernel/arm64/sgemm_direct_arm64_sme1.c should look roughly like

#ifdef HAVE_SME
void CNAME (... 
{
your sme code goes here
}
#else 
void CNAME(...)
{}
#endif

so that the compiler finds something to compile (even if it is an empty function) whether it is running for a target with -march=armv9+sme... temporarily defined or not.
And maybe instead of string comparing the current cpu name in the DYNAMIC_ARCH situation at runtime, we should have a supports_sme() function in driver/others/dynamic_arm64.c that does the AT_HWCAP call (or its alternatives for other platforms) and returns 0 or 1, like the support_avx512 is used in interface/gemm.c for x86_64

[vaiskv]

Got it. I will update the code and push the updated patch. Thanks!

[martin-frbg]

Umm, TARGET=ARMV9SME tells me you're already building on AymenQ's unmerged #5011 ? In that case I might refrain from putting back the HAVE_SME and we could just live with the TARGET name(s) like in the Skylakex sgemm_direct kernel.
(I'm just unsure if the Apple M4 is going to be that representative of other SME-capable Arm64 targets to come)

[martin-frbg]

Needs

#ifndef HWCAP2_SME
#define HWCAP2_SME 1<<23
#endif

in dynamic_arm64.c

[martin-frbg]

Also, kernel/setparam-ref.c will need a small expansion of the ifdef block referring to sgemm_direct - currently this has only the "ifdef x86_64" part, which is why DYNAMIC_ARCH fails to build on all arm64 platforms in CI.

[martin-frbg]

Hmm, seems that your "conflicting" changes in cmake/system.cmake have already been in the develop branch since early december (PR 5003), that part can probably be removed from this PR ?
The other conflict that git cannot resolve automatically appears to be caused by my merging the "thread throttling profile for NEOVERSEV1" into interface/gemm.c - I think it is just the line differences that are throwing the automation off.

[vaiskv]

Resolved merge conflicts in interface/gemm.c file. Also rebased cmake/system.cmake with develop branch.