Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1721 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.2416. Min Act: -0.1700

Data Index: 5254006 (Open Web Text)

Max Activating Token Index: 986

Text #1

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.2329. Min Act: -0.1700

Data Index: 3249482 (Open Web Text)

Max Activating Token Index: 1012

Text #2

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.2200. Min Act: -0.1700

Data Index: 3249493 (Open Web Text)

Max Activating Token Index: 1018

Text #3

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.2195. Min Act: -0.1700

Data Index: 3249496 (Open Web Text)

Max Activating Token Index: 1022

Text #4

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.2141. Min Act: -0.1700

Data Index: 2624788 (Open Web Text)

Max Activating Token Index: 1018

Text #5

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.2092. Min Act: -0.1700

Data Index: 2041704 (Open Web Text)

Max Activating Token Index: 1021

Text #6

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1775. Min Act: -0.1700

Data Index: 2580632 (Open Web Text)

Max Activating Token Index: 1003

Text #7

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1795. Min Act: -0.1700

Data Index: 7093795 (Open Web Text)

Max Activating Token Index: 1020

Text #8

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1814. Min Act: -0.1700

Data Index: 4079127 (Open Web Text)

Max Activating Token Index: 1019

Text #9

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1783. Min Act: -0.1700

Data Index: 3249490 (Open Web Text)

Max Activating Token Index: 992

Text #10

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1816. Min Act: -0.1700

Data Index: 29211 (Open Web Text)

Max Activating Token Index: 1019

Text #11

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1781. Min Act: -0.1700

Data Index: 5890763 (Open Web Text)

Max Activating Token Index: 1021

Text #12

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1678. Min Act: -0.1700

Data Index: 1324626 (Open Web Text)

Max Activating Token Index: 1022

Text #13

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1703. Min Act: -0.1700

Data Index: 8271750 (Open Web Text)

Max Activating Token Index: 1000

Text #14

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1794. Min Act: -0.1700

Data Index: 663513 (Open Web Text)

Max Activating Token Index: 1009

Text #15

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1698. Min Act: -0.1700

Data Index: 1081145 (Open Web Text)

Max Activating Token Index: 985

Text #16

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1663. Min Act: -0.1700

Data Index: 5417496 (Open Web Text)

Max Activating Token Index: 1007

Text #17

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1609. Min Act: -0.1700

Data Index: 5254012 (Open Web Text)

Max Activating Token Index: 1021

Text #18

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1397. Min Act: -0.1700

Data Index: 2655362 (Open Web Text)

Max Activating Token Index: 999

Text #19

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1572. Min Act: -0.1700

Data Index: 7391111 (Open Web Text)

Max Activating Token Index: 998