Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 26 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3092. Min Range: -3.3092

Max Act: 3.3092. Min Act: -0.1700

Data Index: 1276362 (Open Web Text)

Max Activating Token Index: 465

Text #1

Max Range: 3.3092. Min Range: -3.3092

Max Act: 3.2273. Min Act: -0.1700

Data Index: 2103089 (Open Web Text)

Max Activating Token Index: 263

Text #2

Max Range: 3.3092. Min Range: -3.3092

Max Act: 3.2234. Min Act: -0.1700

Data Index: 1276354 (Open Web Text)

Max Activating Token Index: 303

Text #3

Max Range: 3.3092. Min Range: -3.3092

Max Act: 3.1567. Min Act: -0.1700

Data Index: 4460008 (Open Web Text)

Max Activating Token Index: 131

Text #4

Max Range: 3.3092. Min Range: -3.3092

Max Act: 3.1208. Min Act: -0.1700

Data Index: 2103097 (Open Web Text)

Max Activating Token Index: 135

Text #5

Max Range: 3.3092. Min Range: -3.3092

Max Act: 3.1253. Min Act: -0.1700

Data Index: 6757990 (Open Web Text)

Max Activating Token Index: 529

Text #6

Max Range: 3.3092. Min Range: -3.3092

Max Act: 3.1130. Min Act: -0.1700

Data Index: 8681175 (Open Web Text)

Max Activating Token Index: 354

Text #7

Max Range: 3.3092. Min Range: -3.3092

Max Act: 3.1162. Min Act: -0.1700

Data Index: 8681179 (Open Web Text)

Max Activating Token Index: 461

Text #8

Max Range: 3.3092. Min Range: -3.3092

Max Act: 3.1104. Min Act: -0.1700

Data Index: 845801 (Open Web Text)

Max Activating Token Index: 552

Text #9

Max Range: 3.3092. Min Range: -3.3092

Max Act: 3.0805. Min Act: -0.1700

Data Index: 2103085 (Open Web Text)

Max Activating Token Index: 800

Text #10

Max Range: 3.3092. Min Range: -3.3092

Max Act: 3.0922. Min Act: -0.1700

Data Index: 2103087 (Open Web Text)

Max Activating Token Index: 598

Text #11

Max Range: 3.3092. Min Range: -3.3092

Max Act: 3.0809. Min Act: -0.1700

Data Index: 2103092 (Open Web Text)

Max Activating Token Index: 245

Text #12

Max Range: 3.3092. Min Range: -3.3092

Max Act: 3.0593. Min Act: -0.1700

Data Index: 2103095 (Open Web Text)

Max Activating Token Index: 642

Text #13

Max Range: 3.3092. Min Range: -3.3092

Max Act: 3.0320. Min Act: -0.1700

Data Index: 2103093 (Open Web Text)

Max Activating Token Index: 840

Text #14

Max Range: 3.3092. Min Range: -3.3092

Max Act: 3.0429. Min Act: -0.1700

Data Index: 4643739 (Open Web Text)

Max Activating Token Index: 429

Text #15

Max Range: 3.3092. Min Range: -3.3092

Max Act: 3.0077. Min Act: -0.1700

Data Index: 6531271 (Open Web Text)

Max Activating Token Index: 670

Text #16

Max Range: 3.3092. Min Range: -3.3092

Max Act: 3.0050. Min Act: -0.1700

Data Index: 7022788 (Open Web Text)

Max Activating Token Index: 87

Text #17

Max Range: 3.3092. Min Range: -3.3092

Max Act: 3.0087. Min Act: -0.1700

Data Index: 7480067 (Open Web Text)

Max Activating Token Index: 195

Text #18

Max Range: 3.3092. Min Range: -3.3092

Max Act: 3.0005. Min Act: -0.1700

Data Index: 7884615 (Open Web Text)

Max Activating Token Index: 162

Text #19

Max Range: 3.3092. Min Range: -3.3092

Max Act: 2.9979. Min Act: -0.1700

Data Index: 3442743 (Open Web Text)

Max Activating Token Index: 451