Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2867 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7520. Min Range: -1.7520

Max Act: 1.7520. Min Act: -0.1700

Data Index: 3171307 (Open Web Text)

Max Activating Token Index: 489

Text #1

Max Range: 1.7520. Min Range: -1.7520

Max Act: 1.7452. Min Act: -0.1700

Data Index: 5594462 (Open Web Text)

Max Activating Token Index: 83

Text #2

Max Range: 1.7520. Min Range: -1.7520

Max Act: 1.7406. Min Act: -0.1700

Data Index: 914338 (Open Web Text)

Max Activating Token Index: 971

Text #3

Max Range: 1.7520. Min Range: -1.7520

Max Act: 1.7308. Min Act: -0.1700

Data Index: 2905506 (Open Web Text)

Max Activating Token Index: 412

Text #4

Max Range: 1.7520. Min Range: -1.7520

Max Act: 1.7183. Min Act: -0.1700

Data Index: 8296248 (Open Web Text)

Max Activating Token Index: 198

Text #5

Max Range: 1.7520. Min Range: -1.7520

Max Act: 1.7292. Min Act: -0.1700

Data Index: 5049111 (Open Web Text)

Max Activating Token Index: 937

Text #6

Max Range: 1.7520. Min Range: -1.7520

Max Act: 1.7101. Min Act: -0.1700

Data Index: 2589144 (Open Web Text)

Max Activating Token Index: 511

Text #7

Max Range: 1.7520. Min Range: -1.7520

Max Act: 1.7081. Min Act: -0.1700

Data Index: 5396115 (Open Web Text)

Max Activating Token Index: 383

Text #8

Max Range: 1.7520. Min Range: -1.7520

Max Act: 1.7144. Min Act: -0.1700

Data Index: 5781436 (Open Web Text)

Max Activating Token Index: 980

Text #9

Max Range: 1.7520. Min Range: -1.7520

Max Act: 1.7079. Min Act: -0.1700

Data Index: 2270538 (Open Web Text)

Max Activating Token Index: 326

Text #10

Max Range: 1.7520. Min Range: -1.7520

Max Act: 1.6852. Min Act: -0.1700

Data Index: 5813288 (Open Web Text)

Max Activating Token Index: 812

Text #11

Max Range: 1.7520. Min Range: -1.7520

Max Act: 1.6891. Min Act: -0.1700

Data Index: 1447979 (Open Web Text)

Max Activating Token Index: 503

Text #12

Max Range: 1.7520. Min Range: -1.7520

Max Act: 1.6955. Min Act: -0.1700

Data Index: 2728893 (Open Web Text)

Max Activating Token Index: 470

Text #13

Max Range: 1.7520. Min Range: -1.7520

Max Act: 1.6936. Min Act: -0.1700

Data Index: 7792996 (Open Web Text)

Max Activating Token Index: 426

Text #14

Max Range: 1.7520. Min Range: -1.7520

Max Act: 1.6922. Min Act: -0.1700

Data Index: 5070764 (Open Web Text)

Max Activating Token Index: 232

Text #15

Max Range: 1.7520. Min Range: -1.7520

Max Act: 1.6947. Min Act: -0.1700

Data Index: 2739483 (Open Web Text)

Max Activating Token Index: 375

Text #16

Max Range: 1.7520. Min Range: -1.7520

Max Act: 1.6846. Min Act: -0.1700

Data Index: 8675255 (Open Web Text)

Max Activating Token Index: 187

Text #17

Max Range: 1.7520. Min Range: -1.7520

Max Act: 1.6803. Min Act: -0.1700

Data Index: 8181883 (Open Web Text)

Max Activating Token Index: 486

Text #18

Max Range: 1.7520. Min Range: -1.7520

Max Act: 1.6813. Min Act: -0.1700

Data Index: 8640932 (Open Web Text)

Max Activating Token Index: 249

Text #19

Max Range: 1.7520. Min Range: -1.7520

Max Act: 1.6830. Min Act: -0.1700

Data Index: 7590588 (Open Web Text)

Max Activating Token Index: 302