Implementation notes: amd64, oki, crypto_dh/ed521gs

Computer: oki
Architecture: amd64
CPU ID: GenuineIntel-00050654-bfebfbff
SUPERCOP version: 20181123
Operation: crypto_dh
Primitive: ed521gs

Time	Object size	Test size	Implementation	Compiler	Benchmark date	SUPERCOP version
3261484	? ? ?	? ? ?	`ref`	`gcc_-march=native_-mtune=native_-Os_-fomit-frame-pointer_-fwrapv`	20190110	20181123
3389818	? ? ?	? ? ?	`ref`	`gcc_-march=native_-mtune=native_-O3_-fomit-frame-pointer_-fwrapv`	20190110	20181123
3406118	? ? ?	? ? ?	`ref`	`clang_-O3_-fwrapv_-march=x86-64_-mcpu=core-avx2_-mavx2_-maes_-mpclmul_-fomit-frame-pointer_-Qunused-arguments`	20190110	20181123
3439742	? ? ?	? ? ?	`ref`	`clang_-O3_-fwrapv_-mavx2_-fomit-frame-pointer_-Qunused-arguments`	20190110	20181123
3447900	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-m64_-march=nocona_-O3_-fomit-frame-pointer`	20190110	20181123
3448124	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-march=nocona_-O3_-fomit-frame-pointer`	20190110	20181123
3460454	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-m64_-march=barcelona_-O3_-fomit-frame-pointer`	20190110	20181123
3460770	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=native_-mtune=native_-O3_-fomit-frame-pointer`	20190110	20181123
3462746	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-march=barcelona_-O3_-fomit-frame-pointer`	20190110	20181123
3463776	? ? ?	? ? ?	`ref`	`gcc_-march=barcelona_-O3_-fomit-frame-pointer`	20190110	20181123
3463782	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=barcelona_-O3_-fomit-frame-pointer`	20190110	20181123
3463840	? ? ?	? ? ?	`ref`	`gcc_-march=k8_-O3_-fomit-frame-pointer`	20190110	20181123
3466568	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=k8_-O3_-fomit-frame-pointer`	20190110	20181123
3466614	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-march=k8_-O3_-fomit-frame-pointer`	20190110	20181123
3467454	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=nocona_-O3_-fomit-frame-pointer`	20190110	20181123
3468814	? ? ?	? ? ?	`ref`	`gcc_-m64_-O3_-fomit-frame-pointer`	20190110	20181123
3469096	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-m64_-march=k8_-O3_-fomit-frame-pointer`	20190110	20181123
3469616	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=core2_-msse4_-O3_-fomit-frame-pointer`	20190110	20181123
3470412	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=core2_-msse4.1_-O3_-fomit-frame-pointer`	20190110	20181123
3471028	? ? ?	? ? ?	`ref`	`gcc_-Os_-fomit-frame-pointer`	20190110	20181123
3471300	? ? ?	? ? ?	`ref`	`gcc_-fno-schedule-insns_-Os_-fomit-frame-pointer`	20190110	20181123
3472138	? ? ?	? ? ?	`ref`	`gcc_-march=nocona_-O3_-fomit-frame-pointer`	20190110	20181123
3474732	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=corei7_-Os_-fomit-frame-pointer`	20190110	20181123
3475378	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-Os_-fomit-frame-pointer`	20190110	20181123
3475994	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-m64_-Os_-fomit-frame-pointer`	20190110	20181123
3476010	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=core2_-O3_-fomit-frame-pointer`	20190110	20181123
3476656	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=corei7-avx_-Os_-fomit-frame-pointer`	20190110	20181123
3476694	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=barcelona_-Os_-fomit-frame-pointer`	20190110	20181123
3476886	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-fno-schedule-insns_-Os_-fomit-frame-pointer`	20190110	20181123
3477628	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=k8_-Os_-fomit-frame-pointer`	20190110	20181123
3477780	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=core-avx-i_-Os_-fomit-frame-pointer`	20190110	20181123
3478028	? ? ?	? ? ?	`ref`	`gcc_-march=k8_-Os_-fomit-frame-pointer`	20190110	20181123
3478114	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=core2_-msse4_-Os_-fomit-frame-pointer`	20190110	20181123
3478222	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=core2_-msse4.1_-Os_-fomit-frame-pointer`	20190110	20181123
3478736	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=nocona_-Os_-fomit-frame-pointer`	20190110	20181123
3479722	? ? ?	? ? ?	`ref`	`gcc_-march=nocona_-Os_-fomit-frame-pointer`	20190110	20181123
3479990	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=core2_-Os_-fomit-frame-pointer`	20190110	20181123
3480312	? ? ?	? ? ?	`ref`	`gcc_-march=barcelona_-Os_-fomit-frame-pointer`	20190110	20181123
3482332	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=core-avx-i_-O3_-fomit-frame-pointer`	20190110	20181123
3486018	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=core-avx2_-Os_-fomit-frame-pointer`	20190110	20181123
3491506	? ? ?	? ? ?	`ref`	`clang_-O3_-fwrapv_-mavx_-fomit-frame-pointer_-Qunused-arguments`	20190110	20181123
3491688	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-march=k8_-Os_-fomit-frame-pointer`	20190110	20181123
3491798	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-m64_-march=k8_-Os_-fomit-frame-pointer`	20190110	20181123
3491936	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-m64_-march=barcelona_-Os_-fomit-frame-pointer`	20190110	20181123
3492288	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-march=barcelona_-Os_-fomit-frame-pointer`	20190110	20181123
3492672	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-m64_-march=nocona_-Os_-fomit-frame-pointer`	20190110	20181123
3492946	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-march=nocona_-Os_-fomit-frame-pointer`	20190110	20181123
3495482	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=native_-mtune=native_-Os_-fomit-frame-pointer`	20190110	20181123
3505414	? ? ?	? ? ?	`ref`	`clang_-O3_-fwrapv_-mavx_-maes_-mpclmul_-fomit-frame-pointer_-Qunused-arguments`	20190110	20181123
3518428	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=core-avx2_-O3_-fomit-frame-pointer`	20190110	20181123
3527078	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-fno-schedule-insns_-O3_-fomit-frame-pointer`	20190110	20181123
3527330	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-m64_-O3_-fomit-frame-pointer`	20190110	20181123
3528156	? ? ?	? ? ?	`ref`	`gcc_-O3_-fomit-frame-pointer`	20190110	20181123
3528380	? ? ?	? ? ?	`ref`	`gcc_-fno-schedule-insns_-O3_-fomit-frame-pointer`	20190110	20181123
3529004	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-O3_-fomit-frame-pointer`	20190110	20181123
3531256	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=barcelona_-O2_-fomit-frame-pointer`	20190110	20181123
3532464	? ? ?	? ? ?	`ref`	`gcc_-march=barcelona_-O2_-fomit-frame-pointer`	20190110	20181123
3539730	? ? ?	? ? ?	`ref`	`gcc_-march=k8_-O2_-fomit-frame-pointer`	20190110	20181123
3539990	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=k8_-O2_-fomit-frame-pointer`	20190110	20181123
3540124	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=corei7-avx_-O3_-fomit-frame-pointer`	20190110	20181123
3543942	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-march=barcelona_-O2_-fomit-frame-pointer`	20190110	20181123
3544170	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=corei7_-O3_-fomit-frame-pointer`	20190110	20181123
3544384	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-m64_-march=barcelona_-O2_-fomit-frame-pointer`	20190110	20181123
3545700	? ? ?	? ? ?	`ref`	`gcc_-march=native_-mtune=native_-O2_-fomit-frame-pointer_-fwrapv`	20190110	20181123
3550138	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-O2_-fomit-frame-pointer`	20190110	20181123
3550882	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=corei7-avx_-O2_-fomit-frame-pointer`	20190110	20181123
3551064	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-m64_-march=k8_-O2_-fomit-frame-pointer`	20190110	20181123
3552962	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-march=k8_-O2_-fomit-frame-pointer`	20190110	20181123
3553690	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=core2_-msse4_-O2_-fomit-frame-pointer`	20190110	20181123
3560704	? ? ?	? ? ?	`ref`	`clang_-mcpu=cortex-a8_-mfpu=neon_-O3_-fomit-frame-pointer_-fwrapv_-Qunused-arguments`	20190110	20181123
3560798	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=core-avx2_-O2_-fomit-frame-pointer`	20190110	20181123
3560916	? ? ?	? ? ?	`ref`	`clang_-mcpu=native_-mfpu=neon_-O3_-fomit-frame-pointer_-fwrapv_-Qunused-arguments`	20190110	20181123
3585184	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-m64_-O2_-fomit-frame-pointer`	20190110	20181123
3585226	? ? ?	? ? ?	`ref`	`clang_-O3_-fomit-frame-pointer_-Qunused-arguments`	20190110	20181123
3596010	? ? ?	? ? ?	`ref`	`clang_-mcpu=cortex-a9_-mfpu=neon_-O3_-fomit-frame-pointer_-fwrapv_-Qunused-arguments`	20190110	20181123
3596278	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-march=nocona_-O2_-fomit-frame-pointer`	20190110	20181123
3598682	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-m64_-march=nocona_-O2_-fomit-frame-pointer`	20190110	20181123
3602824	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=corei7_-O2_-fomit-frame-pointer`	20190110	20181123
3603364	? ? ?	? ? ?	`ref`	`gcc_-march=nocona_-O2_-fomit-frame-pointer`	20190110	20181123
3604136	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=nocona_-O2_-fomit-frame-pointer`	20190110	20181123
3609762	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-fno-schedule-insns_-O2_-fomit-frame-pointer`	20190110	20181123
3611720	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=core-avx-i_-O2_-fomit-frame-pointer`	20190110	20181123
3612584	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=core2_-O2_-fomit-frame-pointer`	20190110	20181123
3614082	? ? ?	? ? ?	`ref`	`gcc_-fno-schedule-insns_-O2_-fomit-frame-pointer`	20190110	20181123
3614346	? ? ?	? ? ?	`ref`	`gcc_-O2_-fomit-frame-pointer`	20190110	20181123
3614408	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=native_-mtune=native_-O2_-fomit-frame-pointer`	20190110	20181123
3614760	? ? ?	? ? ?	`ref`	`gcc_-m64_-O2_-fomit-frame-pointer`	20190110	20181123
3633192	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=core2_-msse4.1_-O2_-fomit-frame-pointer`	20190110	20181123
3904796	? ? ?	? ? ?	`ref`	`gcc_-m64_-Os_-fomit-frame-pointer`	20190110	20181123
5667148	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-m64_-march=k8_-O_-fomit-frame-pointer`	20190110	20181123
5667500	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-march=k8_-O_-fomit-frame-pointer`	20190110	20181123
5689324	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-march=barcelona_-O_-fomit-frame-pointer`	20190110	20181123
5689588	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-m64_-march=barcelona_-O_-fomit-frame-pointer`	20190110	20181123
5690396	? ? ?	? ? ?	`ref`	`gcc_-march=barcelona_-O_-fomit-frame-pointer`	20190110	20181123
5690646	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=barcelona_-O_-fomit-frame-pointer`	20190110	20181123
5693988	? ? ?	? ? ?	`ref`	`gcc_-march=k8_-O_-fomit-frame-pointer`	20190110	20181123
5694826	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=k8_-O_-fomit-frame-pointer`	20190110	20181123
5765992	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-m64_-O_-fomit-frame-pointer`	20190110	20181123
5777176	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-fno-schedule-insns_-O_-fomit-frame-pointer`	20190110	20181123
5781128	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=core2_-O_-fomit-frame-pointer`	20190110	20181123
5783650	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=nocona_-O_-fomit-frame-pointer`	20190110	20181123
5830054	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-m64_-march=nocona_-O_-fomit-frame-pointer`	20190110	20181123
5830714	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-march=nocona_-O_-fomit-frame-pointer`	20190110	20181123
5831494	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops_-O_-fomit-frame-pointer`	20190110	20181123
5833334	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=core2_-msse4.1_-O_-fomit-frame-pointer`	20190110	20181123
5833428	? ? ?	? ? ?	`ref`	`gcc_-O_-fomit-frame-pointer`	20190110	20181123
5834168	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=core2_-msse4_-O_-fomit-frame-pointer`	20190110	20181123
5834974	? ? ?	? ? ?	`ref`	`gcc_-m64_-O_-fomit-frame-pointer`	20190110	20181123
5835256	? ? ?	? ? ?	`ref`	`gcc_-fno-schedule-insns_-O_-fomit-frame-pointer`	20190110	20181123
5841786	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=corei7_-O_-fomit-frame-pointer`	20190110	20181123
5844184	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=core-avx-i_-O_-fomit-frame-pointer`	20190110	20181123
5851102	? ? ?	? ? ?	`ref`	`gcc_-march=nocona_-O_-fomit-frame-pointer`	20190110	20181123
5852474	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=corei7-avx_-O_-fomit-frame-pointer`	20190110	20181123
5915246	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=native_-mtune=native_-O_-fomit-frame-pointer`	20190110	20181123
5953282	? ? ?	? ? ?	`ref`	`gcc_-march=native_-mtune=native_-O_-fomit-frame-pointer_-fwrapv`	20190110	20181123
5970486	? ? ?	? ? ?	`ref`	`gcc_-m64_-march=core-avx2_-O_-fomit-frame-pointer`	20190110	20181123
13297650	? ? ?	? ? ?	`ref`	`cc`	20190110	20181123
13306994	? ? ?	? ? ?	`ref`	`gcc`	20190110	20181123
13307312	? ? ?	? ? ?	`ref`	`gcc_-funroll-loops`	20190110	20181123

Compiler output

Implementation: ref
Security model: unknown
Compiler: clang -O3 -fwrapv -march=native -fomit-frame-pointer -Qunused-arguments

try.c: fatal error: error in backend: Cannot select: 0x1abcf20: v4i64 = X86ISD::VTRUNC 0x1abcd50
try.c: 0x1abcd50: v16i32 = vselect 0x1aaa5c0, 0x1a61ac0, 0x1abcc20
try.c: 0x1aaa5c0: v4i1 = X86ISD::PCMPGTM 0x1aa3c00, 0x1a38c80
try.c: 0x1aa3c00: v4i64 = X86ISD::VBROADCAST 0x1a39010
try.c: 0x1a39010: i64,ch = load<LD8[%uglygep72]> 0x199edc0, 0x1a65bf0, undef:i64
try.c: 0x1a65bf0: i64 = add 0x1aa4ae0, 0x1a4b5f0
try.c: 0x1aa4ae0: i64 = X86ISD::Wrapper TargetGlobalAddress:i64<[64 x i64]* @cycles> 0
try.c: 0x1a8a5c0: i64 = TargetGlobalAddress<[64 x i64]* @cycles> 0
try.c: 0x1a4b5f0: i64 = shl 0x1a89d70, Constant:i8<3>
try.c: 0x1a89d70: i64,ch = CopyFromReg 0x199edc0, Register:i64 %vreg50
try.c: 0x1aa4fa0: i64 = Register %vreg50
try.c: 0x1a20870: i8 = Constant<3>
try.c: 0x1aa1a60: i64 = undef
try.c: 0x1a38c80: v4i64,ch = CopyFromReg 0x199edc0, Register:v4i64 %vreg13
try.c: 0x1a50c40: v4i64 = Register %vreg13
try.c: 0x1a61ac0: v16i32 = X86ISD::VBROADCAST 0x1a89fd0
try.c: 0x1a89fd0: i32,ch = load<LD4[ConstantPool]> 0x199edc0, 0x1abc890, undef:i64
try.c: 0x1abc890: i64 = X86ISD::Wrapper TargetConstantPool:i64<i32 1> 0
try.c: 0x1a4e760: i64 = TargetConstantPool<i32 1> 0
try.c: 0x1aa1a60: i64 = undef
try.c: 0x1abcc20: v16i32 = BUILD_VECTOR Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>
try.c: 0x1abcaf0: i32 = Constant<0>
try.c: 0x1abcaf0: i32 = Constant<0>
try.c: 0x1abcaf0: i32 = Constant<0>
try.c: 0x1abcaf0: i32 = Constant<0>
try.c: ...

Number of similar (compiler,implementation) pairs: 1, namely:

Compiler	Implementations
clang -O3 -fwrapv -march=native -fomit-frame-pointer -Qunused-arguments	ref

Compiler output

Implementation: ref
Security model: unknown
Compiler: clang -march=native -O3 -fomit-frame-pointer -fwrapv -Qunused-arguments

try.c: fatal error: error in backend: Cannot select: 0x1821020: v4i64 = X86ISD::VTRUNC 0x1820e50
try.c: 0x1820e50: v16i32 = vselect 0x1812950, 0x17cdc20, 0x1820d20
try.c: 0x1812950: v4i1 = X86ISD::PCMPGTM 0x180a920, 0x17ce9d0
try.c: 0x180a920: v4i64 = X86ISD::VBROADCAST 0x17ced60
try.c: 0x17ced60: i64,ch = load<LD8[%uglygep72]> 0x1704dd0, 0x17c3890, undef:i64
try.c: 0x17c3890: i64 = add 0x180b800, 0x17553f0
try.c: 0x180b800: i64 = X86ISD::Wrapper TargetGlobalAddress:i64<[64 x i64]* @cycles> 0
try.c: 0x17aa9f0: i64 = TargetGlobalAddress<[64 x i64]* @cycles> 0
try.c: 0x17553f0: i64 = shl 0x17aa1a0, Constant:i8<3>
try.c: 0x17aa1a0: i64,ch = CopyFromReg 0x1704dd0, Register:i64 %vreg50
try.c: 0x180bcc0: i64 = Register %vreg50
try.c: 0x179b020: i8 = Constant<3>
try.c: 0x1808f90: i64 = undef
try.c: 0x17ce9d0: v4i64,ch = CopyFromReg 0x1704dd0, Register:v4i64 %vreg13
try.c: 0x17b9410: v4i64 = Register %vreg13
try.c: 0x17cdc20: v16i32 = X86ISD::VBROADCAST 0x17aa400
try.c: 0x17aa400: i32,ch = load<LD4[ConstantPool]> 0x1704dd0, 0x1820990, undef:i64
try.c: 0x1820990: i64 = X86ISD::Wrapper TargetConstantPool:i64<i32 1> 0
try.c: 0x17a5700: i64 = TargetConstantPool<i32 1> 0
try.c: 0x1808f90: i64 = undef
try.c: 0x1820d20: v16i32 = BUILD_VECTOR Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>
try.c: 0x1820bf0: i32 = Constant<0>
try.c: 0x1820bf0: i32 = Constant<0>
try.c: 0x1820bf0: i32 = Constant<0>
try.c: 0x1820bf0: i32 = Constant<0>
try.c: ...

Number of similar (compiler,implementation) pairs: 1, namely:

Compiler	Implementations
clang -march=native -O3 -fomit-frame-pointer -fwrapv -Qunused-arguments	ref