Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1322 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0116. Min Range: -3.0116

Max Act: 3.0116. Min Act: -0.1700

Data Index: 6687964 (Open Web Text)

Max Activating Token Index: 155

Text #1

Max Range: 3.0116. Min Range: -3.0116

Max Act: 2.9816. Min Act: -0.1700

Data Index: 1189455 (Open Web Text)

Max Activating Token Index: 370

Text #2

Max Range: 3.0116. Min Range: -3.0116

Max Act: 2.9562. Min Act: -0.1700

Data Index: 403928 (Open Web Text)

Max Activating Token Index: 952

Text #3

Max Range: 3.0116. Min Range: -3.0116

Max Act: 2.9377. Min Act: -0.1700

Data Index: 4349897 (Open Web Text)

Max Activating Token Index: 720

Text #4

Max Range: 3.0116. Min Range: -3.0116

Max Act: 2.9548. Min Act: -0.1700

Data Index: 3696077 (Open Web Text)

Max Activating Token Index: 555

Text #5

Max Range: 3.0116. Min Range: -3.0116

Max Act: 2.9020. Min Act: -0.1700

Data Index: 8205169 (Open Web Text)

Max Activating Token Index: 49

Text #6

Max Range: 3.0116. Min Range: -3.0116

Max Act: 2.9114. Min Act: -0.1700

Data Index: 3658105 (Open Web Text)

Max Activating Token Index: 603

Text #7

Max Range: 3.0116. Min Range: -3.0116

Max Act: 2.8945. Min Act: -0.1700

Data Index: 8234925 (Open Web Text)

Max Activating Token Index: 288

Text #8

Max Range: 3.0116. Min Range: -3.0116

Max Act: 2.8990. Min Act: -0.1700

Data Index: 226305 (Open Web Text)

Max Activating Token Index: 947

Text #9

Max Range: 3.0116. Min Range: -3.0116

Max Act: 2.8977. Min Act: -0.1700

Data Index: 5896124 (Open Web Text)

Max Activating Token Index: 309

Text #10

Max Range: 3.0116. Min Range: -3.0116

Max Act: 2.8961. Min Act: -0.1700

Data Index: 5300166 (Open Web Text)

Max Activating Token Index: 347

Text #11

Max Range: 3.0116. Min Range: -3.0116

Max Act: 2.8662. Min Act: -0.1700

Data Index: 5863962 (Open Web Text)

Max Activating Token Index: 765

Text #12

Max Range: 3.0116. Min Range: -3.0116

Max Act: 2.8984. Min Act: -0.1700

Data Index: 4049556 (Open Web Text)

Max Activating Token Index: 782

Text #13

Max Range: 3.0116. Min Range: -3.0116

Max Act: 2.8686. Min Act: -0.1700

Data Index: 5194207 (Open Web Text)

Max Activating Token Index: 178

Text #14

Max Range: 3.0116. Min Range: -3.0116

Max Act: 2.8757. Min Act: -0.1700

Data Index: 4708303 (Open Web Text)

Max Activating Token Index: 886

Text #15

Max Range: 3.0116. Min Range: -3.0116

Max Act: 2.8860. Min Act: -0.1700

Data Index: 4616981 (Open Web Text)

Max Activating Token Index: 991

Text #16

Max Range: 3.0116. Min Range: -3.0116

Max Act: 2.8848. Min Act: -0.1700

Data Index: 246821 (Open Web Text)

Max Activating Token Index: 654

Text #17

Max Range: 3.0116. Min Range: -3.0116

Max Act: 2.8949. Min Act: -0.1700

Data Index: 317051 (Open Web Text)

Max Activating Token Index: 232

Text #18

Max Range: 3.0116. Min Range: -3.0116

Max Act: 2.8855. Min Act: -0.1700

Data Index: 2711487 (Open Web Text)

Max Activating Token Index: 18

Text #19

Max Range: 3.0116. Min Range: -3.0116

Max Act: 2.8679. Min Act: -0.1700

Data Index: 3601348 (Open Web Text)

Max Activating Token Index: 206