Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 98 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2007. Min Range: -4.2007

Max Act: 4.2007. Min Act: -0.1700

Data Index: 7035183 (Open Web Text)

Max Activating Token Index: 946

Text #1

Max Range: 4.2007. Min Range: -4.2007

Max Act: 4.1596. Min Act: -0.1700

Data Index: 1596028 (Open Web Text)

Max Activating Token Index: 477

Text #2

Max Range: 4.2007. Min Range: -4.2007

Max Act: 4.0651. Min Act: -0.1700

Data Index: 6648630 (Open Web Text)

Max Activating Token Index: 788

Text #3

Max Range: 4.2007. Min Range: -4.2007

Max Act: 4.0517. Min Act: -0.1700

Data Index: 5492554 (Open Web Text)

Max Activating Token Index: 483

Text #4

Max Range: 4.2007. Min Range: -4.2007

Max Act: 3.9865. Min Act: -0.1700

Data Index: 3641601 (Open Web Text)

Max Activating Token Index: 486

Text #5

Max Range: 4.2007. Min Range: -4.2007

Max Act: 4.0143. Min Act: -0.1700

Data Index: 3774672 (Open Web Text)

Max Activating Token Index: 387

Text #6

Max Range: 4.2007. Min Range: -4.2007

Max Act: 3.9744. Min Act: -0.1700

Data Index: 7265222 (Open Web Text)

Max Activating Token Index: 970

Text #7

Max Range: 4.2007. Min Range: -4.2007

Max Act: 3.9538. Min Act: -0.1700

Data Index: 1081654 (Open Web Text)

Max Activating Token Index: 459

Text #8

Max Range: 4.2007. Min Range: -4.2007

Max Act: 3.9687. Min Act: -0.1700

Data Index: 6309007 (Open Web Text)

Max Activating Token Index: 196

Text #9

Max Range: 4.2007. Min Range: -4.2007

Max Act: 3.9479. Min Act: -0.1700

Data Index: 5995516 (Open Web Text)

Max Activating Token Index: 286

Text #10

Max Range: 4.2007. Min Range: -4.2007

Max Act: 3.9550. Min Act: -0.1700

Data Index: 2808157 (Open Web Text)

Max Activating Token Index: 498

Text #11

Max Range: 4.2007. Min Range: -4.2007

Max Act: 3.9367. Min Act: -0.1700

Data Index: 3191503 (Open Web Text)

Max Activating Token Index: 285

Text #12

Max Range: 4.2007. Min Range: -4.2007

Max Act: 3.9256. Min Act: -0.1700

Data Index: 4475651 (Open Web Text)

Max Activating Token Index: 634

Text #13

Max Range: 4.2007. Min Range: -4.2007

Max Act: 3.9043. Min Act: -0.1700

Data Index: 3802284 (Open Web Text)

Max Activating Token Index: 413

Text #14

Max Range: 4.2007. Min Range: -4.2007

Max Act: 3.8412. Min Act: -0.1700

Data Index: 5630455 (Open Web Text)

Max Activating Token Index: 729

Text #15

Max Range: 4.2007. Min Range: -4.2007

Max Act: 3.8694. Min Act: -0.1700

Data Index: 6303755 (Open Web Text)

Max Activating Token Index: 546

Text #16

Max Range: 4.2007. Min Range: -4.2007

Max Act: 3.8491. Min Act: -0.1700

Data Index: 40301 (Open Web Text)

Max Activating Token Index: 835

Text #17

Max Range: 4.2007. Min Range: -4.2007

Max Act: 3.9210. Min Act: -0.1700

Data Index: 32398 (Open Web Text)

Max Activating Token Index: 461

Text #18

Max Range: 4.2007. Min Range: -4.2007

Max Act: 3.8411. Min Act: -0.1700

Data Index: 1446286 (Open Web Text)

Max Activating Token Index: 368

Text #19

Max Range: 4.2007. Min Range: -4.2007

Max Act: 3.8695. Min Act: -0.1700

Data Index: 86016 (Open Web Text)

Max Activating Token Index: 433