Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1588 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.2272. Min Range: -1.2272

Max Act: 1.2272. Min Act: -0.1234

Data Index: 2906275 (Open Web Text)

Max Activating Token Index: 770

Text #1

Max Range: 1.2272. Min Range: -1.2272

Max Act: 1.2120. Min Act: -0.0531

Data Index: 2398999 (Open Web Text)

Max Activating Token Index: 828

Text #2

Max Range: 1.2272. Min Range: -1.2272

Max Act: 1.2044. Min Act: -0.0464

Data Index: 6259677 (Open Web Text)

Max Activating Token Index: 409

Text #3

Max Range: 1.2272. Min Range: -1.2272

Max Act: 1.1947. Min Act: -0.0348

Data Index: 5870050 (Open Web Text)

Max Activating Token Index: 417

Text #4

Max Range: 1.2272. Min Range: -1.2272

Max Act: 1.1944. Min Act: -0.0730

Data Index: 4366162 (Open Web Text)

Max Activating Token Index: 428

Text #5

Max Range: 1.2272. Min Range: -1.2272

Max Act: 1.1857. Min Act: -0.0135

Data Index: 2997787 (Open Web Text)

Max Activating Token Index: 451

Text #6

Max Range: 1.2272. Min Range: -1.2272

Max Act: 1.1863. Min Act: -0.0704

Data Index: 4985861 (Open Web Text)

Max Activating Token Index: 440

Text #7

Max Range: 1.2272. Min Range: -1.2272

Max Act: 1.1849. Min Act: -0.0466

Data Index: 371173 (Open Web Text)

Max Activating Token Index: 428

Text #8

Max Range: 1.2272. Min Range: -1.2272

Max Act: 1.1778. Min Act: -0.0042

Data Index: 7882471 (Open Web Text)

Max Activating Token Index: 835

Text #9

Max Range: 1.2272. Min Range: -1.2272

Max Act: 1.1784. Min Act: -0.0469

Data Index: 8265700 (Open Web Text)

Max Activating Token Index: 445

Text #10

Max Range: 1.2272. Min Range: -1.2272

Max Act: 1.1797. Min Act: -0.0024

Data Index: 8007409 (Open Web Text)

Max Activating Token Index: 437

Text #11

Max Range: 1.2272. Min Range: -1.2272

Max Act: 1.1829. Min Act: -0.0350

Data Index: 1604232 (Open Web Text)

Max Activating Token Index: 448

Text #12

Max Range: 1.2272. Min Range: -1.2272

Max Act: 1.1829. Min Act: -0.0503

Data Index: 6827674 (Open Web Text)

Max Activating Token Index: 414

Text #13

Max Range: 1.2272. Min Range: -1.2272

Max Act: 1.1766. Min Act: -0.1080

Data Index: 8454786 (Open Web Text)

Max Activating Token Index: 767

Text #14

Max Range: 1.2272. Min Range: -1.2272

Max Act: 1.1768. Min Act: -0.0309

Data Index: 3294216 (Open Web Text)

Max Activating Token Index: 458

Text #15

Max Range: 1.2272. Min Range: -1.2272

Max Act: 1.1755. Min Act: -0.0115

Data Index: 5802198 (Open Web Text)

Max Activating Token Index: 414

Text #16

Max Range: 1.2272. Min Range: -1.2272

Max Act: 1.1796. Min Act: -0.0640

Data Index: 6962446 (Open Web Text)

Max Activating Token Index: 401

Text #17

Max Range: 1.2272. Min Range: -1.2272

Max Act: 1.1745. Min Act: -0.0561

Data Index: 789062 (Open Web Text)

Max Activating Token Index: 259

Text #18

Max Range: 1.2272. Min Range: -1.2272

Max Act: 1.1788. Min Act: -0.0282

Data Index: 518136 (Open Web Text)

Max Activating Token Index: 408

Text #19

Max Range: 1.2272. Min Range: -1.2272

Max Act: 1.1772. Min Act: -0.0663

Data Index: 1757560 (Open Web Text)

Max Activating Token Index: 394