NB. ===========================================================================
NB. futhark.ijs - Futhark-J Bridge Library
NB. ===========================================================================
NB.
NB. A bridge between J and Futhark, enabling high-performance parallel array
NB. computations via Futhark's multicore backend.
NB.
NB. VERSION: 1.7.0
NB. LICENSE: MIT
NB.
NB. USAGE:
NB.   load 'path/to/futhark.ijs'
NB.   my_verb =: 'entry foo (xs: []f64) : []f64 = map (*2) xs' fut
NB.   result =: my_verb 1 2 3 4 5
NB.
NB. FEATURES:
NB.   - Compile Futhark code inline in J source files
NB.   - Automatic caching of compiled shared libraries
NB.   - Support for scalar and array inputs and outputs
NB.   - Full support for Futhark data types (i8-i64, u8-u64, f32, f64)
NB.   - Multicore parallel execution
NB.
NB. REQUIREMENTS:
NB.   - Futhark compiler (https://futhark-lang.org/)
NB.   - GCC compiler
NB.   - J 9.x with FFI support (dll addon)
NB.
NB. PUBLIC API:
NB.   fut              - Adverb: compile Futhark code to J verb
NB.   fut_clear_cache  - Clear compiled library cache
NB.   fut_set_threads  - Set thread count for parallel execution
NB.   fut_version      - Display version information
NB.
NB. PLATFORM SUPPORT:
NB.   - Linux (tested)
NB.   - FreeBSD (tested, with automatic patches)
NB.   - macOS (should work, untested)
NB.
NB. ===========================================================================

require 'dll'

NB. ===========================================================================
NB. Configuration
NB. ===========================================================================

NB. FUTHARK_CACHE: Directory for compiled Futhark shared libraries
NB. Libraries are cached by content hash to avoid recompilation
FUTHARK_CACHE =: '/tmp/futhark_j/'

NB. FUTHARK_BACKEND: Compilation backend
NB.   'c'         - Sequential execution (most portable)
NB.   'multicore' - Parallel execution using pthreads
NB.   'ispc'      - Parallel execution using Intel ISPC (requires ispc)
FUTHARK_BACKEND =: 'multicore'

NB. FUTHARK_THREADS: Thread count for multicore/ispc backends
NB. Set to 0 for automatic detection (uses all available cores)
FUTHARK_THREADS =: 12

NB. Ensure cache directory exists
1!:5 :: ] <FUTHARK_CACHE

NB. ===========================================================================
NB. Type Mapping Tables
NB. ===========================================================================
NB.
NB. Futhark and J use different type systems. These functions provide
NB. mappings between Futhark type names and J FFI/memory type codes.
NB.
NB. Futhark Types:
NB.   i8, i16, i32, i64  - Signed integers (8/16/32/64 bit)
NB.   u8, u16, u32, u64  - Unsigned integers (8/16/32/64 bit)
NB.   f32, f64           - IEEE floating point (single/double precision)
NB.
NB. J FFI Type Codes (used with cd):
NB.   c - char (8-bit)
NB.   s - short (16-bit)
NB.   i - int (32-bit)
NB.   x - int64 (64-bit)
NB.   f - float (32-bit)
NB.   d - double (64-bit)
NB.
NB. J Memory Type Codes (used with memr/memw):
NB.   2 - 32-bit integer
NB.   4 - 64-bit integer
NB.   6 - 32-bit float
NB.   8 - 64-bit float (double)
NB. ===========================================================================

NB. fut_type_to_j: Map Futhark type to J FFI type code
NB.   y: Futhark type string (e.g., 'f64', 'i32')
NB.   Returns: Single character J FFI type code
fut_type_to_j =: 3 : 0
  select. y
    case. 'i8'  do. 'c'  NB. signed char
    case. 'i16' do. 's'  NB. short
    case. 'i32' do. 'i'  NB. int
    case. 'i64' do. 'x'  NB. int64
    case. 'u8'  do. 'c'  NB. unsigned char (J treats as signed)
    case. 'u16' do. 's'  NB. unsigned short
    case. 'u32' do. 'i'  NB. unsigned int
    case. 'u64' do. 'x'  NB. uint64
    case. 'f32' do. 'f'  NB. float
    case. 'f64' do. 'd'  NB. double
    case.       do. 'd'  NB. default to double
  end.
)

NB. fut_type_to_memtype: Map Futhark type to J memr/memw type code
NB.   y: Futhark type string
NB.   Returns: Integer type code for memr/memw operations
fut_type_to_memtype =: 3 : 0
  select. y
    case. 'f64' do. 8  NB. double
    case. 'f32' do. 6  NB. float
    case. 'i64' do. 4  NB. int64
    case. 'u64' do. 4  NB. uint64
    case. 'i32' do. 2  NB. int32
    case. 'u32' do. 2  NB. uint32
    case.       do. 8  NB. default to double
  end.
)

NB. fut_type_size: Get byte size for Futhark type
NB.   y: Futhark type string
NB.   Returns: Size in bytes (1, 2, 4, or 8)
fut_type_size =: 3 : 0
  select. y
    case. 'i8';'u8'   do. 1
    case. 'i16';'u16' do. 2
    case. 'i32';'u32';'f32' do. 4
    case. 'i64';'u64';'f64' do. 8
    case. do. 8  NB. default
  end.
)

NB. ===========================================================================
NB. Compilation Pipeline
NB. ===========================================================================
NB.
NB. The compilation pipeline transforms Futhark source code into a shared
NB. library that can be called via J's FFI:
NB.
NB.   1. Hash the source code to generate a unique identifier
NB.   2. Check cache for existing compiled library
NB.   3. Write Futhark source to .fut file
NB.   4. Compile to C using 'futhark <backend> --library'
NB.   5. Apply platform-specific patches (e.g., FreeBSD)
NB.   6. Compile C to shared library using GCC
NB.   7. Return path to .so file
NB.
NB. ===========================================================================

NB. fut_compile: Compile Futhark code to shared library
NB.   y: Futhark source code string
NB.   Returns: Path to compiled .so file, or empty string on failure
NB.
NB. The function uses content-based caching: the same source code will
NB. always produce the same cache key, avoiding redundant compilation.
fut_compile =: 3 : 0
  code =. y

  NB. Generate unique ID using SHA-256 hash (128!:6)
  NB. Take first 16 hex characters for ~64 bits of entropy
  NB. SHA-256 provides cryptographic collision resistance
  hash =. 16 {. 3 (128!:6) code
  id =. 'fut_', hash

  so_file =. FUTHARK_CACHE, id, '.so'
  fut_file =. FUTHARK_CACHE, id, '.fut'
  c_file =. FUTHARK_CACHE, id, '.c'

  NB. Return cached version if exists
  if. fexist so_file do.
    so_file return.
  end.

  NB. Write Futhark source to file
  (code, LF) 1!:2 <fut_file

  NB. Compile Futhark to C
  NB. The --library flag generates a C API instead of a standalone program
  NB. Redirect output to temp file to capture errors even if 2!:0 throws
  err_file =. FUTHARK_CACHE, id, '.err'
  compile_cmd =. 'cd ', FUTHARK_CACHE, ' && futhark ', FUTHARK_BACKEND, ' --library ', id, '.fut >', err_file, ' 2>&1; echo $?'
  compile_result =. 2!:0 :: (''"_) compile_cmd

  if. -. fexist c_file do.
    echo 'Futhark compilation failed:'
    NB. Read error output from file to ensure we capture it
    err_out =. 1!:1 :: (''"_) <err_file
    if. 0 < # err_out do.
      echo err_out
    else.
      echo '(no error output captured)'
    end.
    echo 'Source file: ', fut_file
    '' return.
  end.

  NB. Apply FreeBSD patches for multicore/ispc backends
  NB. Futhark's generated C code doesn't recognize FreeBSD, requiring:
  NB.   1. Missing headers: sys/resource.h, unistd.h
  NB.   2. CPU count detection via sysconf(_SC_NPROCESSORS_ONLN)
  if. (<FUTHARK_BACKEND) e. 'multicore';'ispc' do.
    NB. Add FreeBSD case to num_processors() function
    sed_cmd =. 'sed -i.bak ''s/#elif __EMSCRIPTEN__/#elif defined(__FreeBSD__)\n  return sysconf(_SC_NPROCESSORS_ONLN);\n#elif __EMSCRIPTEN__/'' ', c_file
    2!:0 :: ] sed_cmd
    NB. Add required headers at top of file
    header_cmd =. '(echo "#include <sys/resource.h>"; echo "#include <unistd.h>"; cat ', c_file, ') > /tmp/fut_temp.c && mv /tmp/fut_temp.c ', c_file
    2!:0 :: ] header_cmd
  end.

  NB. Compile C to shared library
  gcc_cmd =. 'gcc -O3 -fPIC -shared -o ', so_file, ' ', c_file, ' -lm -lpthread 2>&1'
  gcc_out =. 2!:0 :: (''"_) gcc_cmd

  if. -. fexist so_file do.
    echo 'GCC compilation failed:'
    echo gcc_out
    '' return.
  end.

  so_file
)

NB. ===========================================================================
NB. Metadata Extraction
NB. ===========================================================================
NB.
NB. These functions parse Futhark source code to extract metadata needed
NB. for FFI calls: entry point name, data type, and return type.
NB.
NB. Futhark entry point syntax:
NB.   entry <name> (<param>: <type>) : <return_type> = <body>
NB.
NB. Examples:
NB.   entry sum (xs: []f64) : f64 = reduce (+) 0 xs
NB.   entry double (xs: []f64) : []f64 = map (*2) xs
NB.
NB. ===========================================================================

NB. fut_parse_entry: Extract entry point name from Futhark code
NB.   y: Futhark source code string
NB.   Returns: Entry point name (string), or empty if not found
fut_parse_entry =: 3 : 0
  code =. y
  if. +./ 'entry ' E. code do.
    NB. Find position after 'entry ' and extract until next space
    rest =. (6 + 'entry ' I.@E. code) }. code
    ({.~ i.&' ') rest
  else.
    ''
  end.
)

NB. fut_parse_dtype: Extract primary data type from Futhark code
NB.   y: Futhark source code string
NB.   Returns: Data type string (e.g., 'f64', 'i32')
NB.
NB. Detection priority: array types first, then scalar types.
NB. Defaults to 'f64' if no recognized type found.
fut_parse_dtype =: 3 : 0
  code =. y
  NB. Check for array types first (most common case)
  if. +./ '[]f64' E. code do. 'f64'
  elseif. +./ '[]f32' E. code do. 'f32'
  elseif. +./ '[]i64' E. code do. 'i64'
  elseif. +./ '[]i32' E. code do. 'i32'
  NB. Check for scalar return types
  elseif. +./ ': f64' E. code do. 'f64'
  elseif. +./ ': f32' E. code do. 'f32'
  elseif. +./ ': i64' E. code do. 'i64'
  elseif. +./ ': i32' E. code do. 'i32'
  NB. Default to f64 (most common in scientific computing)
  elseif. do. 'f64'
  end.
)

NB. fut_returns_array: Check if function returns an array type
NB.   y: Futhark source code string
NB.   Returns: 1 if returns array, 0 if returns scalar
NB.
NB. Examines the return type annotation after the parameter list.
NB. Array types start with '[]', e.g., '[]f64', '[]i32'.
NB. Handles both ') : []' and '): []' patterns.
fut_returns_array =: 3 : 0
  code =. y
  NB. Check for array return type: ') : []' or '): []' patterns
  (+./ ') : []' E. code) +. +./ '): []' E. code
)

NB. fut_count_brackets: Count array dimensions in a type string
NB.   y: Type string (e.g., '[][]f64', '[][32]i64')
NB.   Returns: Number of dimensions (array rank)
NB.
NB. Note: Counts '[' characters, not '[]' pairs, because Futhark
NB. uses [n] for fixed-size dimensions (e.g., '[32]' in '[][32]i64')
fut_count_brackets =: 3 : 0
  +/ '[' E. y
)

NB. fut_parse_return_type: Extract full return type string after ) : for entry point
NB.   y: Futhark source code
NB.   Returns: Return type string (e.g., '[][]f64', 'f64', '[]i32')
NB.
NB. Note: Must handle multi-parameter functions like (xs: []f64) (ys: []f64) : f64
NB. The return type is marked by '):' or ') :' pattern AFTER the 'entry' keyword
fut_parse_return_type =: 3 : 0
  code =. y
  NB. Find 'entry ' and look for return type after it
  entry_pos =. I. 'entry ' E. code
  if. 0 = # entry_pos do. 'f64' return. end.
  NB. Get code starting from entry point
  code_from_entry =. ({. entry_pos) }. code
  NB. Try ') : ' pattern first (with space before colon)
  ret_marker =. I. ') : ' E. code_from_entry
  skip =. 4
  NB. If not found, try '): ' pattern (no space before colon)
  if. 0 = # ret_marker do.
    ret_marker =. I. '): ' E. code_from_entry
    skip =. 3
  end.
  if. 0 < # ret_marker do.
    NB. Skip past the marker to get the type (use FIRST occurrence from entry)
    type_start =. skip + {. ret_marker
    type_rest =. type_start }. code_from_entry
    NB. Find end of type (space before = or newline)
    eq_pos =. I. ' =' E. type_rest
    if. 0 < # eq_pos do.
      ({. eq_pos) {. type_rest
    else.
      ({.~ i.&LF) type_rest
    end.
  else.
    'f64'  NB. default
  end.
)

NB. fut_parse_input_type: Extract input parameter type from first parameter of entry point
NB.   y: Futhark source code
NB.   Returns: Input type string (e.g., '[]f64', '[][]i32')
NB.
NB. Note: Must find the entry point's parameters, not parameters from helper functions
fut_parse_input_type =: 3 : 0
  code =. y
  NB. Find 'entry ' and look for parameters after it
  entry_pos =. I. 'entry ' E. code
  if. 0 = # entry_pos do. '[]f64' return. end.
  NB. Get code starting from entry point
  code_from_entry =. ({. entry_pos) }. code
  NB. Find first '(' after entry
  open_paren =. I. '(' E. code_from_entry
  close_paren =. I. ')' E. code_from_entry
  if. (0 < # open_paren) *. 0 < # close_paren do.
    NB. Extract text between ( and )
    start =. 1 + {. open_paren
    len =. ({. close_paren) - start
    param_section =. len {. start }. code_from_entry
    NB. Find ': ' and extract type until ')' or ','
    colon_pos =. I. ': ' E. param_section
    if. 0 < # colon_pos do.
      type_start =. 2 + {. colon_pos
      type_rest =. type_start }. param_section
      NB. Type ends at ')' or ','
      end_pos =. <./ (# type_rest), (I. ')' E. type_rest), I. ',' E. type_rest
      end_pos {. type_rest
    else.
      '[]f64'  NB. default
    end.
  else.
    '[]f64'
  end.
)

NB. fut_extract_basetype: Extract base type from array type
NB.   y: Type string (e.g., '[][]f64', '[][32]i64', 'f64')
NB.   Returns: Base type without brackets (e.g., 'f64', 'i64')
NB.
NB. Note: Handles both dynamic dimensions '[]' and fixed dimensions '[n]'
fut_extract_basetype =: 3 : 0
  type =. y
  NB. Remove all bracket prefixes (both '[]' and '[n]')
  while. '[' = {. type do.
    NB. Find closing bracket and skip past it
    close_pos =. type i. ']'
    type =. (close_pos + 1) }. type
  end.
  type
)

NB. fut_output_rank: Get rank of output array
NB.   y: Futhark source code
NB.   Returns: Array rank (0 for scalar, 1 for 1D, 2 for 2D, etc.)
fut_output_rank =: 3 : 0
  ret_type =. fut_parse_return_type y
  fut_count_brackets ret_type
)

NB. fut_input_rank: Get rank of input array
NB.   y: Futhark source code
NB.   Returns: Array rank
fut_input_rank =: 3 : 0
  in_type =. fut_parse_input_type y
  fut_count_brackets in_type
)

NB. fut_output_dtype: Get base data type of output
NB.   y: Futhark source code
NB.   Returns: Base type string (e.g., 'f64')
fut_output_dtype =: 3 : 0
  ret_type =. fut_parse_return_type y
  fut_extract_basetype ret_type
)

NB. fut_input_dtype: Get base data type of input
NB.   y: Futhark source code
NB.   Returns: Base type string (e.g., 'f64')
fut_input_dtype =: 3 : 0
  in_type =. fut_parse_input_type y
  fut_extract_basetype in_type
)

NB. ===========================================================================
NB. Multi-Input Parameter Parsing (for dyadic verbs)
NB. ===========================================================================
NB.
NB. Futhark supports multiple input parameters, each in its own parentheses:
NB.   entry f (xs: []f64) (ys: []f64) : f64 = ...
NB.
NB. These functions detect and parse multiple input parameters to enable
NB. dyadic J verbs that take two array arguments.
NB.
NB. ===========================================================================

NB. fut_count_inputs: Count number of input parameters
NB.   y: Futhark source code
NB.   Returns: Number of input parameters (1 or 2)
fut_count_inputs =: 3 : 0
  code =. y
  NB. Find the return type marker ') :'
  ret_pos =. I. ') :' E. code
  if. 0 = # ret_pos do. 1 return. end.
  NB. Count '(' before the return type declaration
  before_ret =. ({. ret_pos) {. code
  +/ '(' E. before_ret
)

NB. fut_parse_input_n: Parse the nth input parameter type (0-indexed)
NB.   x: parameter index (0 or 1)
NB.   y: Futhark source code
NB.   Returns: Type string for that parameter
fut_parse_input_n =: 4 : 0
  n =. x
  code =. y
  NB. Find all '(' positions
  open_parens =. I. '(' E. code
  if. n >: # open_parens do. '[]f64' return. end.
  NB. Get the nth parameter section
  start =. 1 + n { open_parens
  rest =. start }. code
  NB. Find closing ')' for this parameter
  close =. {. I. ')' E. rest
  param_section =. close {. rest
  NB. Extract type after ': '
  colon_pos =. I. ': ' E. param_section
  if. 0 = # colon_pos do. '[]f64' return. end.
  type_start =. 2 + {. colon_pos
  type_start }. param_section
)

NB. fut_input_dtype_n: Get base data type of nth input
NB.   x: parameter index
NB.   y: Futhark source code
fut_input_dtype_n =: 4 : 0
  in_type =. x fut_parse_input_n y
  fut_extract_basetype in_type
)

NB. fut_input_rank_n: Get rank of nth input
NB.   x: parameter index
NB.   y: Futhark source code
fut_input_rank_n =: 4 : 0
  in_type =. x fut_parse_input_n y
  fut_count_brackets in_type
)

NB. ===========================================================================
NB. Context Initialization
NB. ===========================================================================
NB.
NB. Futhark requires a context for all operations. Creating a context has
NB. overhead (thread pool initialization, memory allocation), so we create
NB. it once when the verb is defined rather than on each call.
NB.
NB. This eliminates the need for "warmup" calls before benchmarking.
NB.
NB. ===========================================================================

NB. fut_init_context: Create Futhark context for a compiled library
NB.   y: Path to shared library (.so file)
NB.   Returns: Boxed pair (ctx ; so_file) for use in executors
NB.
NB. The context is created with the current FUTHARK_THREADS setting.
NB. The context persists for the lifetime of the J session.
fut_init_context =: 3 : 0
  so =. y
  cfg =. (so, ' futhark_context_config_new > x') cd ''
  (so, ' futhark_context_config_set_num_threads > n x i') cd cfg; FUTHARK_THREADS
  ctx =. (so, ' futhark_context_new > x x') cd cfg
  NB. Note: We intentionally don't free cfg - it must persist with ctx
  ctx
)

NB. ===========================================================================
NB. FFI Execution - Scalar Returns
NB. ===========================================================================
NB.
NB. Executes a Futhark function that returns a scalar value.
NB.
NB. Futhark C API for scalar returns:
NB.   int futhark_entry_<name>(
NB.     struct futhark_context *ctx,
NB.     <type> *out,
NB.     const struct futhark_<type>_1d *in
NB.   );
NB.
NB. Memory management:
NB.   - Context is pre-created and reused (passed in x)
NB.   - Allocate output memory with mema
NB.   - Read result with memr after sync
NB.   - Free only per-call resources (input array, output memory)
NB.
NB. ===========================================================================

NB. fut_exec_scalar: Execute Futhark function returning scalar
NB.   x: Boxed 6-tuple (so_file ; ctx ; entry_name ; in_dtype ; out_dtype ; in_rank)
NB.   y: Input data array (J numeric array of any rank)
NB.   Returns: Scalar result value
fut_exec_scalar =: 4 : 0
  'so ctx entry in_dtype out_dtype in_rank' =. x
  data =. , y  NB. Flatten input
  out_memtype =. fut_type_to_memtype out_dtype
  out_elemsize =. fut_type_size out_dtype

  NB. Get input shape, padding with 1s if needed
  in_shape =. in_rank {. ($ y), in_rank # 1

  NB. Create Futhark input array - dynamically build signature and args
  new_sig =. so, in_dtype fut_make_new_sig in_rank
  new_args =. (ctx;data) fut_make_new_args in_shape
  arr =. new_sig cd new_args

  NB. Allocate memory for output scalar
  out_mem =. mema out_elemsize

  NB. Call the entry point function
  entry_func =. ' futhark_entry_', entry, ' > i x x x'
  rc =. (so, entry_func) cd ctx; out_mem; arr

  NB. Synchronize to ensure computation is complete
  (so, ' futhark_context_sync > i x') cd ctx

  NB. Read result from allocated memory
  result_val =. 0 { memr out_mem, 0, 1, out_memtype
  memf out_mem

  NB. Cleanup: free only the input array (context persists)
  in_free =. ' futhark_free_', in_dtype, '_', (": in_rank), 'd > i x x'
  (so, in_free) cd ctx; arr

  result_val
)

NB. ===========================================================================
NB. FFI Execution - Array Returns
NB. ===========================================================================
NB.
NB. Executes a Futhark function that returns an array of any dimensionality.
NB.
NB. Futhark C API for array returns:
NB.   int futhark_entry_<name>(
NB.     struct futhark_context *ctx,
NB.     struct futhark_<type>_<rank>d **out,  // pointer-to-pointer
NB.     const struct futhark_<type>_<rank>d *in
NB.   );
NB.
NB. The implementation dynamically builds FFI signatures based on array rank.
NB.
NB. ===========================================================================

NB. fut_make_new_sig: Build FFI signature for futhark_new_<type>_<rank>d
NB.   x: dtype (e.g., 'f64')
NB.   y: rank (e.g., 2)
NB.   Returns: Signature string (e.g., ' futhark_new_f64_2d > x x *d x x')
fut_make_new_sig =: 4 : 0
  dtype =. x
  rank =. y
  jtype =. fut_type_to_j dtype
  NB. Build: ' futhark_new_<dtype>_<rank>d > x x *<jtype> x x ...'
  base =. ' futhark_new_', dtype, '_', (": rank), 'd > x x *', jtype
  base, ; rank # <' x'
)

NB. fut_make_new_args: Build argument list for futhark_new call
NB.   x: (ctx ; data)
NB.   y: shape (dimensions)
NB.   Returns: Boxed argument list (ctx ; data ; d0 ; d1 ; ...)
fut_make_new_args =: 4 : 0
  'ctx data' =. x
  shape =. y
  NB. Build boxed list: ctx, data, then each dimension separately
  (<ctx), (<data), <"0 shape
)

NB. fut_exec_nd: Universal executor for Futhark functions returning arrays
NB.   x: Boxed 7-tuple (so_file ; ctx ; entry ; in_dtype ; out_dtype ; in_rank ; out_rank)
NB.   y: Input data array (J numeric array of any rank)
NB.   Returns: Result array with shape determined by Futhark
fut_exec_nd =: 4 : 0
  'so ctx entry in_dtype out_dtype in_rank out_rank' =. x
  data =. , y  NB. Flatten input to 1D for FFI
  out_memtype =. fut_type_to_memtype out_dtype
  out_elemsize =. fut_type_size out_dtype

  NB. Get input shape, padding with 1s if needed
  in_shape =. in_rank {. ($ y), in_rank # 1

  NB. Create Futhark input array - dynamically build signature and args
  new_sig =. so, in_dtype fut_make_new_sig in_rank
  new_args =. (ctx;data) fut_make_new_args in_shape
  arr_in =. new_sig cd new_args

  NB. Allocate space for output array pointer
  ptr_space =. mema 8
  (0) memw ptr_space, 0, 1, 4

  NB. Call the entry point function
  entry_func =. ' futhark_entry_', entry, ' > i x x x'
  rc =. (so, entry_func) cd ctx; ptr_space; arr_in

  NB. Check for errors
  if. rc ~: 0 do.
    memf ptr_space
    in_free =. ' futhark_free_', in_dtype, '_', (": in_rank), 'd > i x x'
    (so, in_free) cd ctx; arr_in
    echo 'Futhark execution failed with code: ', ": rc
    (out_rank # 0) return.  NB. Return empty array of correct rank
  end.

  NB. Synchronize and get output array handle
  (so, ' futhark_context_sync > i x') cd ctx
  arr_out =. 0 { memr ptr_space, 0, 1, 4
  memf ptr_space

  NB. Query output shape (returns pointer to out_rank int64_t values)
  shape_func =. ' futhark_shape_', out_dtype, '_', (": out_rank), 'd > x x x'
  shape_ptr =. (so, shape_func) cd ctx; arr_out
  out_shape =. memr shape_ptr, 0, out_rank, 4
  out_size =. */ out_shape

  NB. Allocate buffer and copy values from Futhark array
  data_mem =. mema out_size * out_elemsize
  values_func =. ' futhark_values_', out_dtype, '_', (": out_rank), 'd > i x x x'
  (so, values_func) cd ctx; arr_out; data_mem

  NB. Read result into J array
  result_data =. memr data_mem, 0, out_size, out_memtype
  memf data_mem

  NB. Cleanup: free per-call resources (context persists)
  in_free =. ' futhark_free_', in_dtype, '_', (": in_rank), 'd > i x x'
  out_free =. ' futhark_free_', out_dtype, '_', (": out_rank), 'd > i x x'
  (so, in_free) cd ctx; arr_in
  (so, out_free) cd ctx; arr_out

  NB. Reshape result to output shape
  out_shape $ result_data
)

NB. ===========================================================================
NB. FFI Execution - Dyadic (Two-Input) Functions
NB. ===========================================================================
NB.
NB. These executors handle Futhark functions with two input arrays,
NB. enabling dyadic J verbs: x verb y
NB.
NB. Futhark C API for two-input functions:
NB.   int futhark_entry_<name>(
NB.     struct futhark_context *ctx,
NB.     <output> *out,
NB.     const struct futhark_<type1>_<rank1>d *in1,
NB.     const struct futhark_<type2>_<rank2>d *in2
NB.   );
NB.
NB. ===========================================================================

NB. fut_exec_scalar_dyad: Execute two-input Futhark function returning scalar
NB.   x: Boxed tuple (so_file ; ctx ; entry ; in1_dtype ; in1_rank ; in2_dtype ; in2_rank ; out_dtype)
NB.   y: Boxed pair (left_array ; right_array)
NB.   Returns: Scalar result value
fut_exec_scalar_dyad =: 4 : 0
  'so ctx entry in1_dtype in1_rank in2_dtype in2_rank out_dtype' =. x
  'left right' =. y
  data1 =. , left
  data2 =. , right
  out_memtype =. fut_type_to_memtype out_dtype
  out_elemsize =. fut_type_size out_dtype

  NB. Get input shapes
  in1_shape =. in1_rank {. ($ left), in1_rank # 1
  in2_shape =. in2_rank {. ($ right), in2_rank # 1

  NB. Create first Futhark input array
  new_sig1 =. so, in1_dtype fut_make_new_sig in1_rank
  new_args1 =. (ctx;data1) fut_make_new_args in1_shape
  arr1 =. new_sig1 cd new_args1

  NB. Create second Futhark input array
  new_sig2 =. so, in2_dtype fut_make_new_sig in2_rank
  new_args2 =. (ctx;data2) fut_make_new_args in2_shape
  arr2 =. new_sig2 cd new_args2

  NB. Allocate memory for output scalar
  out_mem =. mema out_elemsize

  NB. Call the entry point function with two inputs
  entry_func =. ' futhark_entry_', entry, ' > i x x x x'
  rc =. (so, entry_func) cd ctx; out_mem; arr1; arr2

  NB. Synchronize
  (so, ' futhark_context_sync > i x') cd ctx

  NB. Read result
  result_val =. 0 { memr out_mem, 0, 1, out_memtype
  memf out_mem

  NB. Cleanup input arrays (context persists)
  in1_free =. ' futhark_free_', in1_dtype, '_', (": in1_rank), 'd > i x x'
  in2_free =. ' futhark_free_', in2_dtype, '_', (": in2_rank), 'd > i x x'
  (so, in1_free) cd ctx; arr1
  (so, in2_free) cd ctx; arr2

  result_val
)

NB. fut_exec_nd_dyad: Execute two-input Futhark function returning array
NB.   x: Boxed tuple (so_file ; ctx ; entry ; in1_dtype ; in1_rank ; in2_dtype ; in2_rank ; out_dtype ; out_rank)
NB.   y: Boxed pair (left_array ; right_array)
NB.   Returns: Result array
fut_exec_nd_dyad =: 4 : 0
  'so ctx entry in1_dtype in1_rank in2_dtype in2_rank out_dtype out_rank' =. x
  'left right' =. y
  data1 =. , left
  data2 =. , right
  out_memtype =. fut_type_to_memtype out_dtype
  out_elemsize =. fut_type_size out_dtype

  NB. Get input shapes
  in1_shape =. in1_rank {. ($ left), in1_rank # 1
  in2_shape =. in2_rank {. ($ right), in2_rank # 1

  NB. Create both Futhark input arrays
  new_sig1 =. so, in1_dtype fut_make_new_sig in1_rank
  new_args1 =. (ctx;data1) fut_make_new_args in1_shape
  arr1 =. new_sig1 cd new_args1

  new_sig2 =. so, in2_dtype fut_make_new_sig in2_rank
  new_args2 =. (ctx;data2) fut_make_new_args in2_shape
  arr2 =. new_sig2 cd new_args2

  NB. Allocate space for output array pointer
  ptr_space =. mema 8
  (0) memw ptr_space, 0, 1, 4

  NB. Call the entry point function with two inputs
  entry_func =. ' futhark_entry_', entry, ' > i x x x x'
  rc =. (so, entry_func) cd ctx; ptr_space; arr1; arr2

  NB. Error handling
  if. rc ~: 0 do.
    memf ptr_space
    in1_free =. ' futhark_free_', in1_dtype, '_', (": in1_rank), 'd > i x x'
    in2_free =. ' futhark_free_', in2_dtype, '_', (": in2_rank), 'd > i x x'
    (so, in1_free) cd ctx; arr1
    (so, in2_free) cd ctx; arr2
    echo 'Futhark execution failed with code: ', ": rc
    (out_rank # 0) return.
  end.

  NB. Synchronize and get output array handle
  (so, ' futhark_context_sync > i x') cd ctx
  arr_out =. 0 { memr ptr_space, 0, 1, 4
  memf ptr_space

  NB. Query output shape
  shape_func =. ' futhark_shape_', out_dtype, '_', (": out_rank), 'd > x x x'
  shape_ptr =. (so, shape_func) cd ctx; arr_out
  out_shape =. memr shape_ptr, 0, out_rank, 4
  out_size =. */ out_shape

  NB. Allocate buffer and copy values
  data_mem =. mema out_size * out_elemsize
  values_func =. ' futhark_values_', out_dtype, '_', (": out_rank), 'd > i x x x'
  (so, values_func) cd ctx; arr_out; data_mem

  NB. Read result into J array
  result_data =. memr data_mem, 0, out_size, out_memtype
  memf data_mem

  NB. Cleanup per-call resources (context persists)
  in1_free =. ' futhark_free_', in1_dtype, '_', (": in1_rank), 'd > i x x'
  in2_free =. ' futhark_free_', in2_dtype, '_', (": in2_rank), 'd > i x x'
  out_free =. ' futhark_free_', out_dtype, '_', (": out_rank), 'd > i x x'
  (so, in1_free) cd ctx; arr1
  (so, in2_free) cd ctx; arr2
  (so, out_free) cd ctx; arr_out

  out_shape $ result_data
)

NB. ===========================================================================
NB. FFI Execution - Scalar Input Functions
NB. ===========================================================================
NB.
NB. These executors handle Futhark functions with scalar inputs (not arrays).
NB. Scalar inputs are passed directly to Futhark, not wrapped in array structs.
NB.
NB. Futhark C API for scalar input, array output:
NB.   int futhark_entry_<name>(
NB.     struct futhark_context *ctx,
NB.     struct futhark_<type>_<rank>d **out,
NB.     const <scalar_type> in
NB.   );
NB.
NB. ===========================================================================

NB. fut_exec_scalar_in_array_out: Execute Futhark function with scalar input returning array
NB.   x: Boxed 6-tuple (so_file ; ctx ; entry ; in_dtype ; out_dtype ; out_rank)
NB.   y: Scalar input value
NB.   Returns: Result array
fut_exec_scalar_in_array_out =: 4 : 0
  'so ctx entry in_dtype out_dtype out_rank' =. x
  scalar_in =. {. , y  NB. Ensure we have a scalar
  out_memtype =. fut_type_to_memtype out_dtype
  out_elemsize =. fut_type_size out_dtype
  in_jtype =. fut_type_to_j in_dtype

  NB. Allocate space for output array pointer
  ptr_space =. mema 8
  (0) memw ptr_space, 0, 1, 4

  NB. Call the entry point function with scalar input
  NB. Signature: entry(ctx, out**, scalar_in)
  entry_func =. ' futhark_entry_', entry, ' > i x x ', in_jtype
  rc =. (so, entry_func) cd ctx; ptr_space; scalar_in

  NB. Check for errors
  if. rc ~: 0 do.
    memf ptr_space
    echo 'Futhark execution failed with code: ', ": rc
    (out_rank # 0) return.
  end.

  NB. Synchronize and get output array handle
  (so, ' futhark_context_sync > i x') cd ctx
  arr_out =. 0 { memr ptr_space, 0, 1, 4
  memf ptr_space

  NB. Query output shape
  shape_func =. ' futhark_shape_', out_dtype, '_', (": out_rank), 'd > x x x'
  shape_ptr =. (so, shape_func) cd ctx; arr_out
  out_shape =. memr shape_ptr, 0, out_rank, 4
  out_size =. */ out_shape

  NB. Allocate buffer and copy values from Futhark array
  data_mem =. mema out_size * out_elemsize
  values_func =. ' futhark_values_', out_dtype, '_', (": out_rank), 'd > i x x x'
  (so, values_func) cd ctx; arr_out; data_mem

  NB. Read result into J array
  result_data =. memr data_mem, 0, out_size, out_memtype
  memf data_mem

  NB. Cleanup output array (no input array to free for scalar input)
  out_free =. ' futhark_free_', out_dtype, '_', (": out_rank), 'd > i x x'
  (so, out_free) cd ctx; arr_out

  NB. Reshape result to output shape
  out_shape $ result_data
)

NB. fut_exec_scalar_in_scalar_out: Execute Futhark function with scalar input returning scalar
NB.   x: Boxed 5-tuple (so_file ; ctx ; entry ; in_dtype ; out_dtype)
NB.   y: Scalar input value
NB.   Returns: Scalar result
fut_exec_scalar_in_scalar_out =: 4 : 0
  'so ctx entry in_dtype out_dtype' =. x
  scalar_in =. {. , y  NB. Ensure we have a scalar
  out_memtype =. fut_type_to_memtype out_dtype
  out_elemsize =. fut_type_size out_dtype
  in_jtype =. fut_type_to_j in_dtype

  NB. Allocate memory for output scalar
  out_mem =. mema out_elemsize

  NB. Call the entry point function with scalar input
  NB. Signature: entry(ctx, out*, scalar_in)
  entry_func =. ' futhark_entry_', entry, ' > i x x ', in_jtype
  rc =. (so, entry_func) cd ctx; out_mem; scalar_in

  NB. Synchronize
  (so, ' futhark_context_sync > i x') cd ctx

  NB. Read result
  result_val =. 0 { memr out_mem, 0, 1, out_memtype
  memf out_mem

  result_val
)

NB. ===========================================================================
NB. Main Public Interface - The 'fut' Adverb
NB. ===========================================================================
NB.
NB. The 'fut' adverb is the primary interface for using Futhark from J.
NB. It takes Futhark source code as a string operand and returns a J verb
NB. that executes the compiled Futhark function.
NB.
NB. Why an adverb?
NB.   J verbs cannot return verbs, but adverbs can. The 'fut' adverb
NB.   creates a closure containing the compiled library path and metadata,
NB.   bound to either fut_exec_scalar or fut_exec_array.
NB.
NB. Usage Pattern:
NB.   my_verb =: 'futhark_code' fut
NB.   result =: my_verb input_data
NB.
NB. ===========================================================================

NB. fut: Compile Futhark code and return executable J verb
NB.
NB. SYNTAX:
NB.   verb =: futhark_code fut
NB.
NB. OPERAND:
NB.   futhark_code - String containing Futhark source with entry point
NB.
NB. RESULT:
NB.   A J verb (monadic or dyadic based on Futhark signature) that:
NB.     - Takes J numeric array(s) as input
NB.     - Returns computed result (scalar or array)
NB.
NB. EXAMPLES:
NB.   NB. Monadic: Array transformation (array in, array out)
NB.   double =: 'entry f (xs: []f64) : []f64 = map (*2) xs' fut
NB.   double 1 2 3 4 5    NB. Returns: 2 4 6 8 10
NB.
NB.   NB. Monadic: Scalar reduction (array in, scalar out)
NB.   total =: 'entry f (xs: []f64) : f64 = reduce (+) 0 xs' fut
NB.   total 1 2 3 4 5     NB. Returns: 15
NB.
NB.   NB. Monadic: Scalar input (scalar in, array out)
NB.   factors =: 'entry f (n: i64) : []i64 = ...' fut
NB.   factors 12          NB. Returns: 2 2 3
NB.
NB.   NB. Dyadic: Dot product (two inputs)
NB.   dot =: 'entry f (xs: []f64) (ys: []f64) : f64 = reduce (+) 0 (map2 (*) xs ys)' fut
NB.   (1 2 3) dot (4 5 6)  NB. Returns: 32
NB.
NB. NOTES:
NB.   - Compilation happens immediately when 'fut' is called
NB.   - Context is created once and reused (no warmup needed)
NB.   - First call may take ~1 second for compilation
NB.   - Entry point name in Futhark code is extracted automatically
NB.   - Return type (scalar vs array) is detected automatically
NB.   - Monadic vs dyadic is detected from number of Futhark parameters
NB.
fut =: 1 : 0
  code =. m

  NB. Compile Futhark code to shared library
  so_file =. fut_compile code
  if. 0 = # so_file do.
    echo 'Failed to compile Futhark code'
    ]  NB. Return identity function as fallback
    return.
  end.

  NB. Create Futhark context immediately (no warmup needed later)
  ctx =. fut_init_context so_file

  NB. Extract metadata from source code
  entry =. fut_parse_entry code
  out_dtype =. fut_output_dtype code
  out_rank =. fut_output_rank code
  num_inputs =. fut_count_inputs code

  NB. Handle monadic (1 input) vs dyadic (2 inputs) cases
  if. num_inputs = 1 do.
    NB. Monadic verb - single input
    in_dtype =. fut_input_dtype code
    in_rank =. fut_input_rank code

    NB. Handle scalar input (in_rank = 0) vs array input
    if. in_rank = 0 do.
      NB. Scalar input
      if. out_rank = 0 do.
        NB. Scalar in, scalar out
        (so_file;ctx;entry;in_dtype;out_dtype)&fut_exec_scalar_in_scalar_out
      else.
        NB. Scalar in, array out
        (so_file;ctx;entry;in_dtype;out_dtype;out_rank)&fut_exec_scalar_in_array_out
      end.
    else.
      NB. Array input
      if. out_rank = 0 do.
        NB. Array in, scalar out
        (so_file;ctx;entry;in_dtype;out_dtype;in_rank)&fut_exec_scalar
      else.
        NB. Array in, array out
        (so_file;ctx;entry;in_dtype;out_dtype;in_rank;out_rank)&fut_exec_nd
      end.
    end.

  elseif. num_inputs = 2 do.
    NB. Dyadic verb - two inputs
    in1_dtype =. 0 fut_input_dtype_n code
    in1_rank =. 0 fut_input_rank_n code
    in2_dtype =. 1 fut_input_dtype_n code
    in2_rank =. 1 fut_input_rank_n code

    if. out_rank = 0 do.
      NB. Scalar output dyadic
      params =. so_file;ctx;entry;in1_dtype;in1_rank;in2_dtype;in2_rank;out_dtype
      params&fut_exec_scalar_dyad@:(;/)
    else.
      NB. Array output dyadic
      params =. so_file;ctx;entry;in1_dtype;in1_rank;in2_dtype;in2_rank;out_dtype;out_rank
      params&fut_exec_nd_dyad@:(;/)
    end.

  elseif. do.
    echo 'Unsupported number of inputs: ', ": num_inputs
    ]  NB. Return identity function as fallback
  end.
)

NB. ===========================================================================
NB. Utility Functions
NB. ===========================================================================

NB. fut_clear_cache: Remove all cached compiled libraries
NB.
NB. Use this to force recompilation of all Futhark code, e.g., after
NB. updating Futhark compiler or changing compiler flags.
NB.
NB. SYNTAX:
NB.   fut_clear_cache ''
NB.
fut_clear_cache =: 3 : 0
  files =. 1!:0 FUTHARK_CACHE, '*'
  if. 0 < # files do.
    for_f. {."1 files do.
      1!:55 :: ] <FUTHARK_CACHE, > f
    end.
    echo 'Cleared ', (": # files), ' cached files'
  else.
    echo 'Cache is empty'
  end.
)

NB. fut_set_threads: Set number of threads for Futhark execution
NB.
NB. Affects all subsequent Futhark calls. Has no effect on the
NB. sequential 'c' backend.
NB.
NB. SYNTAX:
NB.   fut_set_threads n
NB.
NB. ARGUMENT:
NB.   n - Number of threads (positive integer)
NB.
fut_set_threads =: 3 : 0
  FUTHARK_THREADS =: y
  echo 'Futhark threads set to ', ": y
)

NB. fut_version: Display library version and configuration
NB.
NB. SYNTAX:
NB.   fut_version ''
NB.
fut_version =: 3 : 0
  echo 'Futhark-J Bridge v1.7.0'
  echo ''
  echo 'Configuration:'
  echo '  Cache directory: ', FUTHARK_CACHE
  echo '  Backend: ', FUTHARK_BACKEND
  echo '  Threads: ', ": FUTHARK_THREADS
  futhark_ver =. 2!:0 :: ('unknown'"_) 'futhark --version 2>&1'
  echo '  Futhark: ', (}: futhark_ver)
)