Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2945 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4966. Min Range: -3.4966

Max Act: 3.4893. Min Act: -0.1700

Data Index: 8107565 (Open Web Text)

Max Activating Token Index: 23

Text #1

Max Range: 3.4966. Min Range: -3.4966

Max Act: 3.4840. Min Act: -0.1700

Data Index: 6959771 (Open Web Text)

Max Activating Token Index: 255

Text #2

Max Range: 3.4966. Min Range: -3.4966

Max Act: 3.4949. Min Act: -0.1700

Data Index: 8674259 (Open Web Text)

Max Activating Token Index: 24

Text #3

Max Range: 3.4966. Min Range: -3.4966

Max Act: 3.4966. Min Act: -0.1700

Data Index: 8251073 (Open Web Text)

Max Activating Token Index: 999

Text #4

Max Range: 3.4966. Min Range: -3.4966

Max Act: 3.4474. Min Act: -0.1700

Data Index: 1579793 (Open Web Text)

Max Activating Token Index: 26

Text #5

Max Range: 3.4966. Min Range: -3.4966

Max Act: 3.4616. Min Act: -0.1700

Data Index: 3886481 (Open Web Text)

Max Activating Token Index: 402

Text #6

Max Range: 3.4966. Min Range: -3.4966

Max Act: 3.4611. Min Act: -0.1700

Data Index: 5855650 (Open Web Text)

Max Activating Token Index: 138

Text #7

Max Range: 3.4966. Min Range: -3.4966

Max Act: 3.4491. Min Act: -0.1700

Data Index: 4617573 (Open Web Text)

Max Activating Token Index: 6

Text #8

Max Range: 3.4966. Min Range: -3.4966

Max Act: 3.4506. Min Act: -0.1700

Data Index: 4302747 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 3.4966. Min Range: -3.4966

Max Act: 3.4474. Min Act: -0.1700

Data Index: 1979119 (Open Web Text)

Max Activating Token Index: 414

Text #10

Max Range: 3.4966. Min Range: -3.4966

Max Act: 3.4342. Min Act: -0.1700

Data Index: 1250809 (Open Web Text)

Max Activating Token Index: 22

Text #11

Max Range: 3.4966. Min Range: -3.4966

Max Act: 3.4166. Min Act: -0.1700

Data Index: 589005 (Open Web Text)

Max Activating Token Index: 7

Text #12

Max Range: 3.4966. Min Range: -3.4966

Max Act: 3.4158. Min Act: -0.1700

Data Index: 4610484 (Open Web Text)

Max Activating Token Index: 407

Text #13

Max Range: 3.4966. Min Range: -3.4966

Max Act: 3.4355. Min Act: -0.1700

Data Index: 2407156 (Open Web Text)

Max Activating Token Index: 106

Text #14

Max Range: 3.4966. Min Range: -3.4966

Max Act: 3.4286. Min Act: -0.1700

Data Index: 2939117 (Open Web Text)

Max Activating Token Index: 167

Text #15

Max Range: 3.4966. Min Range: -3.4966

Max Act: 3.4126. Min Act: -0.1700

Data Index: 5654320 (Open Web Text)

Max Activating Token Index: 9

Text #16

Max Range: 3.4966. Min Range: -3.4966

Max Act: 3.4189. Min Act: -0.1700

Data Index: 2026828 (Open Web Text)

Max Activating Token Index: 952

Text #17

Max Range: 3.4966. Min Range: -3.4966

Max Act: 3.4242. Min Act: -0.1700

Data Index: 6045424 (Open Web Text)

Max Activating Token Index: 78

Text #18

Max Range: 3.4966. Min Range: -3.4966

Max Act: 3.4374. Min Act: -0.1700

Data Index: 6273412 (Open Web Text)

Max Activating Token Index: 946

Text #19

Max Range: 3.4966. Min Range: -3.4966

Max Act: 3.4244. Min Act: -0.1700

Data Index: 6623062 (Open Web Text)

Max Activating Token Index: 402