If anybody is interested: I wrote a little bash and batch script for automating these steps. https://github.com/pxlrbt/move-wsl

Workaround: Create a %UserProfile%\.wslconfig file in Windows and use it to limit memory assigned to WSL2 VM. Example [wsl2] memory=6GB swap=0 localhostForwarding=true This will still consume the entire 6GBs regardless of Linux memory usage, but at least it'll stop growing more than that. Supported settings are documented here.

Assigning more RAM, as suggested in the accepted answer by @sogu, will probably cost you money. You can alternatively solve this problem free-of-charge by assigning swap space instead. This blog post provides a simple few commands to allocate 1G of swap space, which was enough for conda to install most packages in my case: sudo fallocate -l 1G /swapfile sudo chmod 600 /swapfile sudo mkswap /swapfile sudo swapon /swapfile sudo cp /etc/fstab /etc/fstab.bak echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

Method 1: “The Easy Way” This is my preferred method because it is simple. New environments appear automatically (as long as they have ipykernel installed.) As of the time of this writing, nb_conda_kernels does not yet support Python 3.9. This only affects our base environment which we aren’t going to use for any of our work anyway, right? :). Your other environments can use any Python version including 3.9. Initially, your kernel list (under Change kernel) will only show your current environment. To get your other environment kernels to show automatically: 1. First, install nb_conda_kernels in your base environment. Once this is installed any notebook running from the base environment will automatically show the kernel from any other environment which has ipykernel installed. (base)$ conda install nb_conda_kernels

Background Large Language Models (LLMs), such as GPT-175B, are revolutionizing various tasks in natural language processing. However, their substantial size and computational demands pose significant challenges, especially in resource-constrained environments. Addressing these challenges, model compression has emerged as a critical area of research, focusing on transforming resource-intensive models into compact, efficient versions​​. Method Experiment Results The effectiveness of model compression techniques is evaluated using metrics like the number of parameters, model size, compression ratio, inference time, and FLOPs. Benchmarks and datasets are employed to compare the performance of compressed LLMs with their uncompressed counterparts. While significant advancements have been made, there remains a performance gap between compressed and uncompressed LLMs​​. Conclusion This survey presents a detailed exploration of model compression techniques for LLMs, covering methods, metrics, and benchmarks. It emphasizes the need for advanced research in this area to unlock the full potential of LLMs across various applications, providing valuable insights for ongoing exploration​​.

In ~/.ssh/config, add: Host github.com HostName github.com IdentityFile ~/.ssh/id_rsa_github

Background: The Challenge of Over-Parameterization in Deep Learning Deep learning models, especially in practical applications, often use over-parameterized architectures where the number of parameters exceeds the training data size. Notable examples include Transformer models for language tasks and wide residual networks for computer vision. Despite their high capacity for training data fitting, these models pose challenges in terms of training time and generalization capability​​. The crux of the problem lies in the optimization landscape of these over-parameterized models, typically non-convex, which hampers straightforward analysis and optimization. This issue brings to the fore two key theoretical properties: the convergence gap and the generalization gap, both pivotal for model optimization and generalization​​. Method: Introducing PL Regularization for Model Optimization In a recent study by Chen et al., a novel approach is presented, leveraging the Polyak-Łojasiewicz (PL) condition in the training objective function of over-parameterized models. This approach is grounded in the theoretical analysis showing that a small condition number (the ratio of the Lipschitz constant and the PL constant) implies faster convergence and improved generalization​​. PL Regularized Optimization: The method adds the condition number to the training error, aiming to minimize it through regularized risk minimization. This involves both the PL constant (µ) of the network and the Lipschitz constant \(L_f\)​​. The Polyak-Łojasiewicz (PL) condition is a concept borrowed from optimization theory and has significant implications in the training of over-parameterized models, particularly in deep learning. Let's break down its application and implementation in detail: Understanding the PL Condition What is the PL Condition? The PL condition is a mathematical property that…

Step 1 Open terminal and check cuda version The CUDA version in my system is 12.3. However, in Windows 11, we are unable to install the matching version of the CUDA toolkit. Instead, we will need to install a lower version, such as 12.2. Step 2 Google Cuda toolkit 12.2 and follow the instructions to install Install Success

Problem: In the English version of Windows 10, when displaying Chinese characters, the fonts appear to be incorrect, with some characters appearing larger or smaller, or in odd font styles. Solution For Windows 10 1803 or above Enable UTF-8 language support Control Panel - Region and Language - Region, Non-Unicode (non-Unicode) option.

Background The paper by Chen et al. introduces a novel framework, Only-Train-Once (OTO), which significantly simplifies the neural network pruning process. Traditional pruning methods often involve multi-stage training, are heuristic, and require fine-tuning to reach optimal performance. OTO, on the other hand, compresses full neural networks into slimmer architectures in a single pass, maintaining competitive performance and significantly reducing the computational cost (FLOPs) and model parameters​​. Method The key to OTO's approach lies in two novel concepts: Zero-Invariant Groups (ZIGs): The network's parameters are divided into these groups. Pruning these zero groups does not affect the network's output, thus enabling efficient one-shot pruning. This approach is adaptable to various neural network architectures, including complex ones like residual blocks and multi-head attention mechanisms​​. Half-Space Stochastic Projected Gradient (HSPG): This is a new optimization method that addresses the structured-sparsity optimization problem. It surpasses traditional proximal methods in promoting group sparsity while maintaining comparable convergence rates. The uniqueness of HSPG is its capability to induce sparsity more effectively in deep neural networks (DNNs)​​. The Half-Space Stochastic Projected Gradient (HSPG) is a novel optimization method introduced by Chen et al. in their paper on the Only-Train-Once framework. To understand HSPG, let's break it down into its fundamental concepts and how it functions within the context of neural network training and pruning: Fundamental Concepts Structured Sparsity: HSPG is designed to induce structured sparsity in neural networks. Structured sparsity is about making entire sets of parameters (like filters or neurons) zero, as opposed to unstructured sparsity, where individual weights are set to zero. This is beneficial…