About

What is KnowLM

The KnowLM project
KnowLM is an open-source language model framework for knowledge enhancement, designed to assist researchers and developers in better managing updates and inaccuracies in large models by providing a flexible and customizable toolkit and releasing corresponding models.

The technology, capabilities and content of KnowLM are as follows:

  • Knowledge Prompting
  • Knowledge Editing
  • Knowledge Interaction
Learn More

Knowledge Prompting.

KnowPrompt.

KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction.  WWW2022.

"Knowledge-informed Prompt Learning" seeks to enhance pre-trained models' understanding of specific domains and tasks by incorporating domain-related knowledge, such as knowledge graphs and text examples, through "Knowledge as Prompt".

This approach aims to unlock the full potential of pre-trained models and improve their performance significantly.

For More

Knowledge Editing.

EasyEdit.

Editing Large Language Models: Problems, Methods, and Opportunities. EMNLP2023.

Model Editing, aims to adjust an initial base model's behavior on the particular edit descriptor efficiently without influencing the model behavior on other samples.

Model Editing has strong connections with following topics:updating and fixing bugs for large language models, locating knowledge in large language models and lifelong learning, unlearning.

For More

Knowledge Interaction.

TRICE.

Making Language Models Better Tool Learners with Execution Feedback. NAACL2024.

Knowledge Interaction refers to the dynamic knowledge exchange and feedback between models and humans, environment, tools, or other models.

It is an important process to realize compositional multi-tool learning, complex planning, multi-agent intelligence, and large language model embodied cognition.

For More

Technologies for training LLMs

The full-scale pre-training code (providing conversion, construction, and loading of large corpora) and LoRA instruction fine-tuning code are open-sourced (support multi-machine multi-GPU). Provide examples for knowledge prompt, editing and interaction (still developing).

Github

Resources

KnowLM Zoo

KnowLM Model Zoo

  • We have currently open-sourced the base model of ZhiXi-13B and the extraction model of ZhiXi-13B, including the upcoming release of the base model KnowLM-7B based on LLaMA2.

Portfolio

What we've done

Paper Lists

Related works regarding knowledge prompt, editing and interaction.

  • All

KnowPrompt

Knowledge-aware prompt-tuning approach with synergistic optimization for relation extraction.

EasyEdit

An easy-to-use framework to edit LLMs including GPT-J, Llama1/2, GPT-NEO, GPT2, T5.

KnowAgent

Knowledge-Augmented Planning for LLM-Based Agents.

RetroPrompt

Constructs an open-book knowledge-store from training instances and implements a retrieval mechanism for prompt learning.

RAP

A model-agnostic approach that utilizes schema and knowledge from human-annotated and weakly-supervised data as prompts.

TRICE

A two stage end-to-end framework that enables the model to continually learn through feedback derived from tool execution.