INNER CODE UNIT · Python

reduce_amount

adamyodinsky/TerminalGPT · terminalgpt/chat.py:158

        reduce_amount = self.__total_usage - self.__token_limit
        tokenized_message = []

        while self.exceeding_token_limit():
            message = self.__messages.pop(1)
            tokenized_message = self.__tiktoken_encoder.encode(message["content"])
            while reduce_amount > 0 and len(tokenized_message) > 0:
                self.__total_usage -= 1
                reduce_amount -= 1
                tokenized_message.pop(0)
            if len(tokenized_message) == 0 and self.exceeding_token_limit():
                reduce_amount -= 4  # every message follows <im_start>{role/name}\n{content}<im_end>\n
                self.__total_usage -= 4
                for key, _ in message.items():
                    if key == "name":  # if there's a name, the role is omitted
                        reduce_amount += 1  # role is always required and always 1 token
                        self.__total_usage += 1
        if len(tokenized_message) > 0:

View source record →