Status: [UNTESTED] Endpoints found: http://127.0.0.1:8080/v1/jobs Source: https://github.com/AngshulMajumdar/Tensor-Pruning-API # Tensor pruning for LLM query projections A training-free Python API, CLI, and HTTP job service for global-budget query pruning. The selector minimizes the weighted mixed `(2, 4, 8)` residual norm under one scalar retention budget across all supplied projections. The CPU kernel uses OpenMP and partition selection with expected linear work at fixed hierarchy depth. ## Install Linux or Colab with Python 3.10+, NumPy, and a C++17 compiler with OpenMP. The native kernel compiles once per process on first use. ```sh python -m pip install . # Add the optional HTTP server and its test dependencies. python -m pip install '.[http,test]' ``` ## Python ```python from tensor_pruning import prune_queries # Each projection has shape [query_heads, channels_per_head, input_width]. # For a contiguous Llama-style q_proj matrix, reshape [output, input] as below. q0 = q0_matrix.reshape(num_query_heads, head_dim, q0_matrix.shape[-1]) q1 = q1_matrix.reshape(num_query_heads, head_dim, q1_matrix.shape[-1]) m0, m1 = prune_queries( [q0, q1], keep=K, input_second_moments=[v0, v1], threads=4, scratch_dir="/content", ) ``` Pass one vector of calibration input second moments per projection. Omit the vector for unit moments. Inputs may have different shapes and float dtypes. The input array API accepts native contiguous float16, float32, float64, and registered bfloat16 arrays. The file API accepts standard NumPy float16, float32, and float64 `.npy` tensors. Inputs are read-only from the selector's perspective. Masks contain exactly `K` true slots, including any selected pre-existing zeros. Exact score ties favor the larger flat index in manifest order. Floating-point rounding can affect near ties. ## Memory-mapped CLI Place a JSON manifest and C-contiguous `.npy` arrays under a data directory. The directory may contain sharded projections with different shapes. ```json { "keep": 1000000, "projections": [ { "name": "model.layers.0.self_attn.q_proj",